Spreading the word

NI4AI will be featured on an industry podcast as part of a series on data analytics in November. The podcast will feature an interview with Theo Laughner, a veteran of the industry and NI4AI contributor with expertise in power quality. We’ll share a link to the podcast when it comes out. In the meantime, here’s an article we wrote for the Energy Central blog.

ARPA-E funded platform is democratizing grid data

The National Infrastructure for Artificial Intelligence (AI) on the Grid — or NI4AI — is a three-year ARPA-E initiative designed to enable breakthroughs in data analytics for the grid.

In the past, limited data access has created barriers to advancing the analytical capabilities of the industry. NI4AI is geared at catalyzing the rapid development and deployment of AI tools to improve every aspect of the grid. The project is taking a three pronged approach:

1. The platform offers a high-performance environment that makes it easier to work with large volumes of time series data, and to collaborate.
2. The data captures different aspects of grid behavior relevant to developing analytical tools to address problems on the grid.
3. The community includes data analysts and practitioners working to find new opportunities for data to solve practical problems for utilities.

The project is led by a tech startup, PingThings, and leverages a platform they developed called PredictiveGrid^TM. The platform is used in production by utilities and by researchers, and is being available through NI4AI to make it easier for a broader community of data analysts to work with large volumes of data. The University of California, Berkeley is also a collaborator, and is developing educational content and tutorials to demonstrate different techniques analysts can use to get started.

Data analytics can improve every aspect of the grid

Aging infrastructure, climate risks, cyber threats, renewables, and changing loads make situational awareness essential to maintaining a safe and reliable grid. These changes on the grid will lead to situations that operators have not experienced in the past. For example, they may introduce new dynamics and transients that conventional measurement systems like SCADA and AMI simply aren’t fast enough to capture.

Real-time information and high-frequency sensor data can allow grid operators to detect and respond to emerging risks before they become problematic. Innovative solutions using AI have demonstrated that real-time data from AMI, phasor measurement units (PMUs), and new continuous point-on-wave sensors can alert decision-makers to potential issues, allowing them to take more timely and more effective actions.

One barrier to extracting value from sensor data is that the volume is simply too great. Analytical and visualization tools are necessary to synthesize raw data into information about the system that can help decision-makers figure out how to respond. But analyzing terabytes of data poses a computational challenge for most researchers and practitioners. The NI4AI platform was built by computer scientists to make these large amounts of data easily accessible to practitioners so that they can more readily analyze data to answer questions that come up on the job.

Three pillars of NI4AI: Data, Platform, and Community

The Data

The platform provides open access data from dedicated NI4AI sensors, previously collected grid data, and anonymized utility data “scrubbed” to remove sensitive information. These data support the development of new analytical tools to address issues on the grid by connecting analysts with the data they need to detect problems.

Algorithms are only as good as the data that train them, and some use cases may require more training data than a single utility possesses. Data shared via the platform can lead to higher quality solutions that benefit the industry as a whole.

Open access data can also be used to compare different solutions to the same problem. Transparency in evaluating which algorithms work and which don’t can minimize duplicative work across utilities, and allows analysts to build on one another’s ideas rather than starting from scratch. Open access data can allow analysts to more readily discuss data, share code, and compare results. NI4AI will be hosting hackathons and competitions to encourage knowledge exchange and collaboration among data analysts that are using the platform.

The Platform

The data volume from modern sensor networks requires high-performance infrastructure and tooling. Data historians commonly in use today simply weren’t developed with AI and machine learning in mind. Also, most tools don’t address measurement data quality, such as missing or erroneous measurements. NI4AI is developing standardized and automated mechanisms to clean data and provide feedback about data quality in real time.

Central to NI4AI is a novel, high-performance platform to ingest, store, cleanse, compress, visualize, and process sensor data for AI applications. The project leverages a commercially available platform called PredictiveGridTM, which is used in production by utilities to allow decision-makers to work more effectively and efficiently with big data. The platform capitalizes on best practices established in other industries. Specifically, the platform is purpose-built for high volumes and high-resolution time series data, and is optimized to minimize computing resources to rapidly read and write data. This eliminates many of the computational barriers that have traditionally made it difficult for analysts in the industry to get the most out of time series data on a daily basis.

The Community

Anyone who works in energy appreciates the grid’s complexity. Operating the grid involves making choices that come with inevitable trade-offs. Prioritizing environmental sustainability may come at a cost to reliability. Mitigating wildfire risk may mean causing blackouts. For the industry to use data to inform better, more timely decisions about when to take one action versus another, the industry needs practitioners who really understand the system to become skilled data analysts as well.

NI4AI is facilitating knowledge exchange between grid experts about results that can be achieved using data analytics. The project is hosting data competitions and hackathons focused on addressing specific challenges on the grid. The project is also offering workshops and tutorials to build awareness about new opportunities for using data, and about the solutions that have already been demonstrated.

By streamlining data transfer, code-sharing, and collaboration across institutions, the platform can also streamline the process of establishing collaborations among a diverse community of experts in power systems, data science, and software development.

Becoming an agent for change in the industry

NI4AI will enable industry professionals to use data more effectively in their jobs. Practitioners can support this transition by using the platform, participating in training sessions, contributing data, hosting a sensor pilot, or sharing insights about analytical tools that would be valuable to them on the job. Readers can learn more by signing up at ni4ai.org, or can reach out directly to info@ni4ai.org.

Experts agree that digitalization is just beginning to hit the industry. Developing the skills to work with big data will help practitioners become change agents in their organizations. NI4AI is offering professional development workshops on data visualization and analytics geared at training practitioners to use big data on the job. Look for sessions hosted by CIGRE, NASPI, and IEEE SmartGridConn this fall. Go to ni4ai.org/events to learn more.