The Electric Grid (Desperately) Needs Distributed Systems Experts!
Tl;dr - After 30 years, I’m leaving computer networking to work on electric power grid networks – and you should too!
I have made the tough decision to leave the field of computer networking which has been my career, profession and passion for the last 30 years. To riff on James Hamilton’s insightful 2009 talk “The Data Center Network Is In My Way”, today/fifteen years later the network is rarely in anyone’s way. I and many others have worked for decades to make the network more reliable, scalable, and easier to operate. In that time, networking has changed from a “top-five problem for everyone” to a “non-top-five problem for most”; which is a reasonable metric of success. Sure there are interesting things going on in AI training networks but there are a LOT of people working on that problem already. But my passion has always been about solving the biggest pain points for the most people and it’s clear–to me at least–that computer networking will no longer provide the biggest bang for that particular buck.
Looking around, it’s hard to find a problem that’s a bigger pain to more people than modernizing the global electric grid. After decades of stable load, a raft of system wide-changes are coming down (distributed energy resources, AI workloads, electric cars, etc.) that are forcing a completely different set of requirements. Electric grid operators have a number of unique challenges coming down the pipe including layers of regulations, a challenging economics model, and low ability to say “no” to new loads. A critical difference from computer networks is that the electrical grid must in real-time balance the load on the system versus. the amount of generation. In other words, unlike computer networking, buffers (read: capacitors and batteries) are expensive and currently not pervasively deployed. Specifically, the “last mile” part of the grid that’s closest to your home/store/factory – called the ‘distribution’ grid is experiencing the biggest growing pains.
Electric Distribution Grid Modernization Challenges:
The Distribution Network Scaling: Currently 1 in 3 new cars sold in California is an electric vehicle but the distribution network was never made to handle this load. As a result, it’s becoming more common for neighbors to lose power because they overload their local transformer because too many people charge their EVs at the same time. The effect is compounded as more people move from gas heat to electric heat, e.g., a heat pump. Due to increased demand, there can be multi-year long wait times to upgrade electrical connections, e.g., deploy new EV charging stations, blocking on the electric company to deploy new capacity and equipment.
Control Systems Scaling: Electric companies typically manage 10s of electrical generators, e.g., nuclear power plants, gas powered, etc., but in the modern day they’re being forced to manage orders of magnitudes more devices as people add solar, wind, and battery storage–known as Distributed Energy Resources (DER) to the grid. DER penetration varies by country (thousands to hundred of thousands), but into the future this could easily become millions or hundreds of millions of devices! Existing electrical control systems were not designed for–nor are operators trained to think at–this scale. Open standard control protocols for electric resources, e.g., IEEE 2030.5, are starting to emerge and be deployed but they need to be evolved with operational learnings and technology changes.
Instrumentation/Monitoring: Depending on/if the SmartMeter technology deployed, it can be hours to days before the electric company detects a distribution grid problem in your neighborhood. This affects not only time to repair but also makes new allocation/provisioning decisions slow, static, and conservative.
Assumes “No Warning”, “No Control” Consumption: Historically, you could increase your local electric consumption, e.g., run the dish washer, without coordinating with the electric company. But modern grids are starting to support load-aware technologies like time-based pricing to incentivize off-peak usage, “demand response” technologies where the electric company can dynamically reduce your load (e.g., turn down your electric heater for a bit), or even signaling protocols to warn and bid for a burst of load, e.g., for an 250 Kilo-Watt per-car electric vehicle supercharging site.
Retail Prices Are Skyrocketing: The amount I pay for peak electricity in California has jumped from $0.18 per Kilo-Watt-hour in 2017 to $0.51 in 2024 or almost 3x in seven years! While these costs are ahead of the U.S. national average, so is the EV adoption rate and there is causality. Worse, even these prices do not produce enough budget to upgrade the distribution grid, so the short fall must be made up with smarter generation, demand, and control logic.
And if–like me–you’ve been working in distributed systems, networking, cloud computing, or any related field, all of these problems should sound eerily familiar to you. Because the network grid is becoming a real-time distributed system on a scale and complexity it was never designed for.
Chief Product Officer - SwitchDin
As a result, I’m happy to announce that I’ve decided to join the Australia-based electric grid control software company SwitchDin as their Chief Product Officer. Australia is leading the world in deployment of Distributed Energy Resources (DER) and SwitchDin is leading Australia in managing/scaling them (more details in future posts!). I really feel lucky to have found them and rejoin my old friend Mario Vecchio: it’s a great team, a great set of products, and hopefully as you can tell from this post - a space I’m very passionate about that very much needs help!
