Strategic challenge, a cross-functional program among departments, and for some organizations, an incredible upheaval, the Data Lake is an essential prerequisite to achieve the promise of data sharing and creation of new value at the enterprise scale.
While the Data Lake has become an indispensable lever for information systems, establishing its state of the art is now a challenge, given the varied use cases, environments, and organizational setups. It's a challenge that must be tackled to harness the full potential of one of the best responses from IT departments to corporate strategies.
Initial Promises of the Data Lake
« As a driver of cultural change, the Data Lake compels organizations to create a common heritage. »
Now 3 to 4 years old, the initial drivers of Data Lakes have been of three types:
- Regulatory obligations, where it had become too costly or complex to comply using traditional technologies and architectures,
- Reporting capabilities, whose acceptance was accelerated by the emergence of new Data Visualization practices and the opportunity for storage and computation embodied by the Data Lake,
- Exploration lever, identified prerequisite for the valorization of data to be shared tomorrow by all departments of the company.
The initial successes of Data Lakes were achieved through regulatory uses, enabled by easy access to data from various operational information systems. Leveraging the implemented reporting tools and data availability, reporting uses expanded, calling for the need to secure access and share a business vision on the data.
While the question of ROI of Data Lakes was quickly overshadowed by regulatory objectives, it remained relevant in all other situations. Faced with the human and technological investment made by companies, they were entitled to expect the promised values quickly.
Assessment of Data Lake experiences
« Our management was convinced that data would not be valued to its full potential without significant investment from the company »
The values of the Data Lake can be seen from three angles: Data, Foundation, and Uses.
Far from being independent, these values contribute synergistically to the overall value of the Data Lake. This can be assessed on a case-by-case basis for each Data Lake, where over time data and uses have multiplied on a foundation that often now reaches technological maturity.
« Budgets are easier to find for regulatory projects, which reduce the use of the Data Lake as a data silo warehouse, poorly suited for data cross-referencing and sharing. »
However, the initial promise of data sharing and value creation for the entire company is rarely achieved. It is clear that the Data Lake has more often been seen as a technological opportunity to reduce project time to market. In this perspective, project after project, the structuring of data around a business vision and the pursuit of value at the enterprise scale has not been achieved.
