Those who worked at Berkeley Lab in October 2019 experienced something that nobody ever thought they would encounter: the complete electrical shutdown of every building on the hill site due to a PG&E Public Safety Power Shutoff. The unprecedented event had a silver lining: it highlighted vulnerabilities to be addressed in the Lab’s electrical infrastructure.

“The primary goal of the electrical system, from a broader perspective, is to ensure the continuity of scientific operations by maintaining a reliable power supply.“

– Cezary Jach, Program Director

As the oldest lab in the national lab system, Berkeley Lab’s electrical infrastructure has grown and expanded over the last 90+ years to meet the changing demands of the Lab’s research mission. Following the PSPS events, Lab leadership established a task force to assess electrical vulnerabilities and recommend improvements. Enhancements were made at both the Lab and by PG&E. Over time, the focus shifted from protecting the Lab from utility outages to safeguarding against internal electrical failures, leading to the implementation of the Power System Stewardship Program (PSSP).

“The PSSP’s goal is to enhance the Lab’s electrical system resilience,” explains Cezary Jach, the program director. “This program focuses on the systematic renewal, expansion, and maintenance of our electrical infrastructure.” The new program, which sits within the Facilities Engineering Department, is a cross-functional program envisioned as a best practice to improve power system resilience and the prioritization of funding efforts for maintaining, modernizing, and renewing the Lab’s aging electrical infrastructure.

Cezary, who joined the Lab in 2022, brought with him valuable experience from his work at the Stanford Linear Accelerator, where he developed power resilience programs for major science institutions like SLAC, CERN, and Fermilab. “The number one reason I came to Berkeley Lab,” he said, “was because the PSPS events highlighted that we were vulnerable from a perspective of shutting down the entire lab. After the PSPS events concluded, addressing the vulnerabilities in our power system became critical, leading to the initiation of a new Power System Stewardship Program.”

The PSSP focuses on three related functional areas: reliability, redundancy, and renewal. Together, these three areas create a virtual loop where improvements in each area enable and accelerate improvements in the others, gradually increasing the overall power resilience of the Lab. Some of the key projects which will have significant future impact include:

• Creating a risk-based maintenance program to replace the Lab’s current cyclical maintenance plan. A risk-based approach prioritizes worker safety and ensures that maintenance efforts are targeted in a graded approach with an optimum mix of reactive, interval-based, condition-based, and proactive maintenance practices.
  Designing and installing a comprehensive site-wide Supervisory Control and Data Acquisition (SCADA) system. This system will facilitate remote operation of circuit breakers, enable automatic load shedding, and support rapid power restoration following unplanned or planned outages. Additionally, it will incorporate artificial intelligence to predict electrical faults, among other advanced features.
  Developing a smart grid aimed at balancing power needs during outages. Currently, during unplanned outages, most buildings have limited standby emergency power to support critical loads, preventing unsafe conditions and the loss of scientific data and samples. However, during planned and unplanned outages, the transition between standby and utility power can lead to power losses, posing a risk of equipment damage. The program seeks to develop an interconnected smart grid of generation sources that can seamlessly transfer and share power during both planned and unplanned outages, enhancing system resilience.
  Reconfiguring the power system architecture is already underway. The existing system design places multiple power feed lines within the same vault, a common practice when these systems were originally constructed. This configuration necessitates shutting down all circuits within the vault for maintenance. By redesigning these systems, maintenance, and repairs can be conducted without necessitating large-scale outages.

Cezary Is confident that these and other projects within the PSSP program will help ensure future continuity of power and continuity of science. As Cezary explained, “The primary goal of the electrical system, from a broader perspective, is to ensure the continuity of scientific operations by maintaining a reliable power supply.”