“And the reason it’s hard is that while people tend to think that there’s a computer system on the back end, the system that makes the claim, it’s not really a system. There are layers of systems that date back to the 1980s, and they’ve sort of been, you know, accrued one over the other, over the decades. Some people have called them layers of paint, because it’s true that if you paint on something too many times, those layers of paint will start to crack. And that’s what’s happening.” — Jen Palkha, Founder of Code for America

Almost a decade ago, my brother Drew and I had a dream. California was in the midst of the worst drought in over a millenia, and there was a similar scarcity in recent, locally relevant and rigorous measurement of what was actually working to increase water conservation. We dreamt of using data science practices common in the technology industry, as well as increasingly in major cities like New York and the federal government, to pioneer the future of water management. Joone Kim-Lopez, General Manager / CEO of Moulton Niguel Water District, boldly championed that vision and turned that dream into reality by launching the California Data Collaborative, initially a pilot with a small group of utilities that has since grown by over 5x.

With aging infrastructure and the new normal of climate change, that dream of optimizing every drop of water is more important than ever. Last fall I started a new job as the Regulatory and Public Affairs Manager at my hometown water utility, Crescenta Valley Water District (CVWD). It has been a humbling and eye opening experience to see the data challenges of a water utility from the inside. Notably there are many, many more opportunities for deploying existing data science practices and improving operations across the utility than just water efficiency.

The CaDC has focused its first decade on urban end user water data, pulling customer level water usage and ancillary data to empower research and analytics that improve planning, financial and water resources decisions. Yet phase two of the initial Memorandum of Understanding signed by the founding group of water utilities envisioned integrating a larger universe of data, examining system data to help optimize the energy water nexus and operations across the utility. Working at CVWD, it has become increasingly clear that there is a utility-wide need for the type of digital transformation enabled by the CaDC. Below summarizes the opportunity for the CaDC in several water data verticals across utility operations.

Water Quality Data

The underlying data challenges in water quality mirror the impetus for the CaDC. Currently our District’s water quality data is fragmented across field reports recorded via pencil and paper, our Aveva Historian SCADA system, and the lab results from our water samples made available via a web portal. Those data are integrated through a series of very manually intensive processes involving many excel spreadsheets and transcribing paper records.

See above for the type of table that’s generated from pen and paper records of field observations.

That record keeping is just one of many. Each checkbox is a daily data entry. The pen points to just one of those many. Those are collated and pulled across a complex array of spreadsheets, access databases, and bespoke enterprise IT systems. The pH monthly report is just one of many such daily updates.

Our goal is to digitize the field data collection and automate the integration of those disparate sources. The dream is to make our monthly report submitted to SWRCB DDW an automated output — an easy button similar to the CaDC UWUO service. In addition to lowering the cost of regulatory compliance, this type of integration will enable improved operational metrics and free staff to more proactively manage the foundational mission of ensuring a safe and clean water supply. With increased regulatory burdens and aging infrastructure, improving data is critical.

Here the CaDC network can play a powerful role in amplifying success stories, scaling solutions and serving as a force multiplier.

Modernizing Billing Systems

Looking deeper into the core CaDC water data stack of water usage data, there is an acute need to improve the landscape of water utility billing systems. I have never met a water utility staff member who is happy with their billing system — at best it’s a begrudging acceptance of the status quo as a necessary evil to keep the lights on. Too often these systems cost way too much for too often truly terrible technology products. This hard reality of water billing systems mirrors a larger challenge with enterprise systems of record in government. Jen Palka, the founder of Code for America and former White House public technology appointee, puts the dynamic well:

“It’s not just that they’re spending millions more than they need to because the contracts are so complex, the services and licenses so opaque, and the business practices so aggressive. It’s what they spend on people, and what they simply don’t get done, because they don’t have the right tools for the job.”

A chat GPT generated exagerrated image of old utility billing systems. 

This type of system migration is a vexing challenge. Getting things wrong is a high profile blunder that can result in wrong bills being charged and irate customers. Yet in every challenge there is an opportunity. Here the water industry can build on the larger government technology movement. The Gates and Omidyar Foundations have invested millions in open source billing systems in India that might be redeployed in California. There are growing number of public technology vendors in the ecosystem grown around Code for America that might be tapped into. Institutions like the US Digital Services / 18F marketplace empowering smaller, open source and agile technology providers might be utilized in the water sector.

Here the CaDC could play a powerful role in building those bridges and partnerships for larger and more transformational impact in the core CaDC water data stack. Water utilities can save money and improve business operations at the same time, potentially by orders of magnitude.

Groundwater Recharge

CVWD has partnered with the CaDC, FMWD, GWP and PWP to scope and develop a proposal for an open source planning tool for green infrastructure projects. This project aims to integrate remote sensing and open data to quantify the water recharge potential gains and water efficiency savings of potential projects. The slides below provide an overview:

Google drive link

Recharging water into groundwater basins is just part of the process of increasing local water supplies. That groundwater is adjudicated through local water masters who determine the safe pumping yield from the basin. Groundwater records have a similar situation to water quality, water efficiency and really much of water utility data. Records exist in paper, excel spreadsheets and other systems.

Here the CaDC again has a powerful role to play in enabling water managers to learn from each other about the nuts and bolts of digital transformation. There can often be cultural barriers to shifting to modern digital tools, particularly in potentially sensitive areas like groundwater rights. The CaDC has the right network to help work through those barriers and also potentially prototype solutions, set data standards and accelerate digital transformation.

The Path Ahead

Opportunity and potential must also be tempered with the reality that time and money are finite quantities. The growth of the CaDC over the past ten years and potential for even more dramatic growth across its core water data stack and opportunities to strategically support broader water data transformation demand clear strategic thinking.

The organization’s powerful model of a data science nonprofit led by and for water managers has a unique role to play in the larger water data movement beyond water efficiency. CaDC is excellent at prototyping solutions, enabling data systems integration and supporting the larger water data movement.

Utilities in the CaDC network have also launched inspiring initiatives outside the core water efficiency data vertical. Generative artificial intelligence has the potential to dramatically accelerate the ability of water managers to ask and answer water resources, operational, and other management questions across their utility. In addition, that new technology has great potential to help unlock the wealth of non-quantitative data residing in old pdf water master plans, feasibility studies and other reports over the years. 

The CaDC has tremendous opportunity in front of it, a vast frontier and evolution that echoes the initial founding of the organization. The initial MOU envisioned building new water data infrastructure, a digital public good echoing the visionary public works that the water industry pioneered. The challenges of climate change and aging infrastructure are even more acute today. Let’s rise to the challenge once again with the courage and clear eyed conviction that has made this industry great.

Patrick Atwater was the first Project Manager of the California Data Collaborative.