Trust imagination: ESG data and setting a vision for change
The next stage of progress is transformation with the rise of the data lakehouse
I’ve written about it several times, but in case you missed it, most ESG disclosure regulations drive comparable data for financial services to conduct analyses. According to my LinkedIn poll last week, most sustainability professionals will focus on this area next year.
But where do we go after we get past disclosures and are left with this valuable data? Are companies doomed to continually pass the value of ESG data to investors and regulators without ever recognizing its business value?
Alas, the collective imaginations of our management teams remain fixed!
Per the second edition of the Kyndryl Sustainability Barometer:
Only 19% of organizations have implemented concrete, data-driven sustainability initiatives. This reveals a significant gap between intent and action. Just 6% of organizations lead the way, effectively leveraging sustainability data as a business asset.
Leveraging sustainability data as a business asset could define the sustainability office's next steps in transitioning to a strategic driver of business value. Extrapolating from BSR’s The CSO at a Crossroads report, the Integrated Strategist and the Transformative Change Agent will need data to succeed. CSOs know this but may lack the technology expertise to figure out how to drive change.
The CSOs we interviewed generally appreciate and embrace the fact that the bar has been raised for delivery of business value as well as the achievement of broader impact, smarter and consistent use of data, and improved business integration.
Beyond disclosures, ESG data has business value, but like any sustainability initiative, achieving it involves a collaborative effort, a little technology know-how, and a vision.
Why it matters: ESG data has business value and can inform resilient and transformative strategies, as the status quo is no longer sufficient.
The problem: No business applications exist that integrate ESG, and legacy data platforms are not equipped to handle this.
Sustainability is a new data silo, and we’ve locked it into the compliance workstream and siloed analysis.
Integration might not be as far off as you think, but it requires a mindset shift.
Data collection and the resulting silo
Let’s face it: ESG and corporate sustainability professionals collect mountains of data. Some of it is activity data that informs new context about the company’s operations, like in the case of carbon accounting or value chain operations, and some of it is new data like climate risk or external ESG analysis.
Mostly though, this data remains locked in a silo collected by and used by the sustainability office. I once heard a story of a Chief Sustainability Officer taking the company’s carbon data to the board and stating how they needed to reduce emissions. The board couldn’t figure out how or why because all that was presented was the carbon emissions numbers. There was no context and no business case to change.
What a missed opportunity!
Sustainability offices are uniquely positioned to break down their data silo to convince the business to adopt new, sustainable models and change. Think about the relationships created by going around the company, hat in hand, to collect activity data. Those relationships can be strengthened with new intent focused on what to do now that the company has interpreted the data through an ESG lens.
The management team needs ESG data in the business context because they likely do not recognize how ESG data can inform decision-making across the business from an additive perspective. Consider how sales and marketing can be improved if valid and updated sustainability credentials are leveraged in storytelling or how operations can be enhanced by surfacing efficiency gaps.
The sustainability office must connect the dots for everyone.
Today, many companies are early on in this journey because of the focus on disclosures. Sustainability data is locked in a silo, and likely, a lot of other business applications and datasets are siloed, making connecting the dots difficult. Compounding this are two other significant challenges:
Sustainability offices do not have the organizational influence to convince leaders to modernize the company’s data.
Sustainability offices are understaffed and under-budgeted. They have no means to initiate a data modernization project.
But don’t despair! There are already relationships created through data collection, so you may be ahead of the game. In addition, business units may already be doing some data work. Still, if you want to be that Transformative Change Agent, we need to recognize just how transformative ESG can be to the business, learn a little about technology, and keep the company moving forward.
The prolific data swamps that bog down business
Wouldn’t it be great if all of the company’s business applications had an ESG component? It would make creating business use cases for sustainability work so much easier. That just isn’t reality, and even if it were, the data across those applications would likely be out of date so fast that it would create chaos.
Unless we want to wait for application developers to add ESG context to every business system and map data across clumsily, we need to abstract out the data from our sustainability tools and other business applications and build a new business data platform.
The IT department is the custodian of the company’s technology stack, including data, cybersecurity, and business-focused applications. Depending on the size of your company, they likely run whatever environmental accounting software you’ve selected.
They do not exist to create and maintain data siloes, but they manifest regardless. While cybersecurity concerns may be one reason for data siloes, it is likely business units are keeping their purpose-built application data controlled for the purposes it was meant to serve.
For example, a payment system is meant to process payments, not measure or report the carbon per transaction. Ergo, the payments system only processes payments.
What happens when another business unit wants access to the payments data? Well, IT has solved this. Traditionally, other business units may connect directly to it through the application or back-end APIs, or the data might get exported, copied, and placed somewhere else, abstracting the application layer entirely for some other purpose.
When these data access requests first came from the business, IT initially used a data warehouse, but as data became more complex and varied, it couldn’t keep up. A new idea, the data lake, was created to deal with this growing and varied business data. Storing it was easy, but accessing it proved challenging. From a technology perspective, data lakes have several limitations, including data reliability and performance and poor security and governance controls. They are not meant for daily business users to run analytics against.
Data lakes are often copied and then synced later along purpose-built functions. The way sustainability offices collect data illustrates this challenge. Activity data may be copied from business systems to a data lake and ingested into a carbon accounting tool. There could be multiple data lakes per business unit and likely multiple copies for multiple purposes. For example, the sustainability office might want spend-based information on suppliers to calculate emissions. The operations team also wants those same numbers for inventory analysis. Procurement might be the custodian of that data because it resides in their systems, and they have contracts to manage. That would result in two copies of the original Procurement dataset.
What’s worse is that you need an updated copy whenever you want to analyze the data anew. As a result, data lakes have become these vast data swamps, and the potential of abstracting out the application for democratized analysis has never been fully realized.
One copy of the data for every purpose
Purpose-built apps are good for what they’ve done, and even data warehouses and lakes have served their time. Still, businesses need something more, and we are at a unique time to solve a need that the company might not be considering. After all, the operating environment that businesses find themselves in has changed around our area of expertise.
Consider the way modern business has been digitized since the late 1990s. Digital technology began to offload, streamline, and modernize business processes and systems in a way that seems mundane upon reflection because of just how slowly the transformation was. ESG is a similar force that has shifted the business world and continues to.
So, even if your ERP system has an ESG component, your marketing, sales, financial, operational, and human resources systems should also have one. All relevant data must be consistent, as we don’t want to replicate data and the management plane all over.
Enter application abstraction (again) and a relatively new technology idea: the data lakehouse.
A data lakehouse provides a unified platform for storing a single copy of unstructured and structured data, processing and analyzing data of different types and formats while ensuring data quality, performance, security, and compliance.
Companies currently focus on a data lakehouse as the single destination to facilitate robust machine learning, generative AI, and other mature analytics. Chances are, someone at your company is already considering a data lakehouse for these functions. Yet, it can also be a key solution for companies to leverage and reintegrate ESG data alongside these other datasets without disrupting the business applications.
While this solution fulfills the promise of the original data warehouse and lakes, it isn’t a light lift. Some SaaS platforms, like Microsoft Fabric (where I work, in case you forgot), can be that data lakehouse. Fabric allows for data ingestion, cleansing, engineering, data science, visualization, and those AI use cases in one platform. Again, there is one copy of the data, not copies upon copies like a data lake. This allows for data consistency, performance gains, and cost savings from reduced data movement.
It also means the data for the sustainability office, operations team, and Finance will be the same as in our previous example.
One of my favorite and under-discussed features of a solution like Fabric is its ability to democratize data access through familiar applications like Excel. Data engineers can also connect and work with the data, publish visualizations, clean datasets for self-service analysis, and even create new low-code application experiences.
As this new technology advances across companies, it is timely for sustainability offices, but just understanding its benefits isn’t enough.
The power of the sustainability office to drive transformational change
Getting to a state where you start breaking down data siloes, starting with sustainability data, requires a thoughtful approach. Broadly scoped data modernizations are challenging, but getting on the path with sustainability data as a starting point doesn’t have to be.
Leverage relationships to find budget and influence
Start collecting activity data in the same way you might already be doing. Focus on core business units and add sales, marketing, and finance. These are the groups with budgets and influence.
Schedule time to better understand these groups and their motivations, and be clear about your intent to help build resilience through an additional perspective.
(Optional) If you have enough data, start manually connecting datasets in Excel and see what insights you can uncover.
Define objectives and requirements
Return to the business units with your findings or hypotheses and open the dialogue. Listen and learn.
If you can agree on requirements, define objectives clearly and begin to pursue the means to bring together the data. Secure buy-in to pursue a project from the right business stakeholders.
With the objectives and requirements, it is time to take your idea to IT. Find a friendly data scientist to help be your unofficial champion if possible. Again, they may already have done some work with data lakehouses, so it benefits you to uncover what that work is and align with it.
From here, your role is likely sustainability subject matter expert and keeper of the objectives. With IT’s help, identify what it would take and set the vision. Here’s what IT will need to do:
Design the architecture - chances are this is only the first sustainability data project you will need to undertake. These projects will set up a new way of working with data for your company, so IT will need to develop a scalable architecture.
Ingest the data - Start with the use case. IT will need the sustainability data inside whatever application you’ve selected and data from various business sources within the business unit.
Implement data governance and security - This is a critical step for IT, which must work to ensure the security of datasets.
Develop data processing and analytics workflows - You might re-engage at this stage until the end. Present the use cases here and work closely with the data engineers.
Test, validate, and train users - Once analytics platforms and dashboards are created, work with IT through their User Acceptance Testing processes.
Monitor and optimize - IT should monitor and adjust the environment based on your and the business unit’s feedback.
Scale, evolve, and improve - If the initial project is successful, consider other areas of the business with new datasets, explore new business units, and find new use cases.
In the end, you will hopefully have an impactful proof of concept that can be repeated with various business units and their data. You will also have done something to help the business—organize its data in a new way to deal with new challenges.
I spoke with a sustainability professional the other day about our field, and she mentioned that we need to do something new. Reflecting on her comment, I considered that businesses don’t have applications to deal with the new world of climate change, geopolitical unreset, stakeholders, accountability, and wild uncertainty. Companies can’t deal with these interconnected ESG issues with internal data siloes, and integration of ESG data throughout the business might be just the thing!