How a SCADA to EMS integration setup works in practice

A timestamp without timezone information can shift a valid forecast before it reaches the model. We have completed two integrations, with a third in progress, and the same obstacles have appeared each time. Data formats, EMS write-back, and latency between the meter and model require more hands-on work than vendor API documentation implies. If you expect a simple API-in, API-out exercise, examine these details first.

First friction point: normalising data at ingestion

Distribution operators generally provide 15-minute interval metering data, but its structure depends on the metering vendor, aggregator, and age of the SCADA installation. We have encountered timestamps in local time without timezone annotation, 15-minute averages labelled as interval-start vs interval-end without documentation, and energy values in MWh when the model expects MW average power.

Each of these issues is manageable, but each also needs a normalisation mapping that is specified and checked before the forecast model runs. Vendor documentation often skips this step because the expected input format is obvious from the vendor's side. For the operator, however, the existing SCADA export is the authoritative format because the rest of the stack already consumes it.

For pilot onboarding, we ask for 3 weeks of historical interval data before beginning model training. That gives us time to audit the format. We then write a normalisation adapter per operator, not a universal format parser. The purpose is to map the operator's existing data, rather than require a new export schema. This adds 2-3 days to setup, but avoids training on silently mis-aligned timestamps.

Second friction point: returning forecasts to the EMS

Extracting operational readings from SCADA is usually the simpler part. Sending an external forecast back into the EMS is more involved, and it is often where operators first encounter the gap between an available interface and a usable integration.

Most EMS platforms provide import interfaces, but these were commonly built for single-point forecasts or for the EMS vendor's own forecasting module. A probabilistic forecast with P10, P50, P90 per 15-minute interval per feeder, covering 48 hours, brings a data structure and volume that many import interfaces do not handle naturally. We typically represent this as three separate time-series objects per feeder, one per quantile. The data can be written this way, but the display layer must understand that the objects form one forecast family. Many EMS visualisation tools have no native concept of a forecast band.

During the first integration, we spent 2 weeks with the operator's EMS vendor to add the band representation to the monitoring interface. The issue was not the ability to write the data. The display configuration simply had no template for it. The EMS vendor was cooperative, but this was genuinely new territory for their support team. We ended with a custom display config: P90 as a highlighted threshold line, P50 as the primary forecast trace, and P10 as a lower reference. Duty engineers found the visual useful, although reaching that point took longer than connecting the raw data.

Third friction point: delay in the real-time data path

Latency is the third issue. The eZap model updates every 15 minutes when new metering data arrives, but the time between a smart meter aggregator and the model input is determined by several handoffs. The operator does not always control all of them, and their timing is not always documented.

In one case, the metering aggregator used a 20-minute buffer before pushing interval data to the SCADA system, batching small packets for efficiency. As a result, the "real-time" data path carried a systematic 20-minute lag. A model update triggered at 14:15 could therefore be working with data from 13:55 at best. That delay is not operationally significant across a 48-hour horizon, but it matters for intraday scheduling within a 2-hour window.

The remedy was straightforward. The operator negotiated a shorter flush interval with the aggregator, reducing lag to approximately 5 minutes. The delay had not appeared in the documentation, so we had to test it directly by comparing timestamps on received data against actual meter reads from the field. We now make that timestamp comparison a standard part of integration validation before starting intraday update cycles.

A realistic timeline for the integration

Based on two completed integrations and a third in progress, this is the timeline we use for connecting eZap to a live SCADA and EMS environment. The figures are elapsed calendar days from first data access, rather than engineering hours. They describe a small operator with a standard SCADA installation, not a bespoke multi-substation topology.

Data audit and normalisation adapter: 3 to 5 days. Historical backfill and initial model training: 5 to 7 days. Forecast output injection into EMS and display validation: 5 to 14 days. Latency testing and real-time path validation: 2 to 4 days. The total, including buffer for the EMS vendor, is 3 to 5 weeks from first data access to a live probabilistic forecast feed.

This schedule exceeds what a pure API integration might imply, with coordination from the EMS vendor creating the widest variation. When the EMS team is responsive and familiar with third-party forecast injection, the work is closer to 3 weeks. When the EMS was installed 12 years ago and the vendor has since been acquired twice, the process can take longer.

The integration work still underway

We now have a repeatable process for the normalisation adapter and EMS injection, so those are solved problems in operational terms. The remaining open problem is automation. Each integration still needs some manual setup for the normalisation mapping and EMS display configuration. We are building a configuration layer intended to reduce that work from days to hours for standard SCADA and EMS platforms. It will not cover every edge case, but it will cover the most common ones. This is not a claim that every integration can be standardised. It is the practical software work that determines whether a tool can be used at scale.