Feature engineering for feeder demand forecasting

Cyclical encoding improved day-ahead P50 MAPE by approximately 0.6 percentage points across six feeders. Spot prices added essentially nothing for residential feeders. We cover the signals that helped, the features we dropped, and the test we use before keeping an input.

Calendar and temporal inputs

Calendar and temporal variables produced the most reliable gains in our feeder models. The core set is hour of day, day of week, week of year, and national/regional holiday flags. Together, these inputs explain much of the recurring variation in residential and commercial demand. Anyone working with 15-minute interval data soon sees the difference between Tuesday at 09:00 and Sunday at 09:00, or between Christmas week and the corresponding week in March.

Encoding matters, especially for a gradient-boosted model. Treating hour of day as a linear value from 0 to 23 does not represent the circular relationship between the end and start of the day. We compared that approach with cyclical encoding, using sine and cosine transforms of the hour fraction. On the same feeder dataset, cyclical encoding improved day-ahead P50 MAPE by approximately 0.6 percentage points across six feeders.

Holiday features need a calendar specific to the country and region. We include Portuguese national holidays + regional and municipal holidays in northern Portugal, since local observances can change demand, particularly on light-commercial feeders. The calendar is represented by a binary feature plus a categorical holiday type feature: national holiday, regional holiday, bridge day. This distinguishes the different demand reductions associated with each category.

Temperature and weather inputs

Temperature was the single most important non-calendar feature for residential and mixed feeders in a temperate climate. Its demand relationship is non-linear, with changes around heating and cooling activation thresholds. We therefore use the raw temperature and effective temperature, defined as the deviation from feeder-specific temperature neutrality point (temperature at which HVAC demand is minimal). The neutrality point varies by feeder based on building stock characteristics.

Beyond temperature, we tested five weather variables: wind speed, cloud cover, relative humidity, dew point, solar radiation. Solar radiation mattered for feeders with high commercial occupancy, including offices with large glazed facades and solar-sensitive HVAC systems. On mainly residential feeders, it added only modest improvement beyond the information already represented by temperature.

Wind speed helped coastal feeders where wind-chill creates heating demand beyond the temperature signal. It was not significant for inland northern Portugal feeders, so we dropped it. Relative humidity and dew point had some value during summer heat events, when high humidity + high temperature drives more intense AC load, but their improvement was small and inconsistent.

Demand history and lag inputs

Autoregressive target lags are central to our time-series forecasts. For a 24-hour ahead forecast, the key references are t-24h, the matching period on the prior day, t-168h, the same weekday-hour combination on the prior week, and the matching period on the prior two weeks. These three form our base lag features.

Shorter lags, t-1h, t-2h, t-4h, help with intraday forecasting. For day-ahead work, however, they can add noise by bringing in current-day demand that may not reflect conditions 24 hours later. We enable these lags for intraday updates and disable them for day-ahead planning runs.

We also calculate the mean and standard deviation of the prior 7 days at the same hour-of-day. These rolling statistics improved day-ahead accuracy on feeders with high demand volatility. The mean describes the recent baseline, while the rolling standard deviation estimates short-term variability and correlates with the width of the probabilistic band model should output.

Inputs we tested and removed

We tested load metering data quality flags from SCADA, including indicators for intervals where values had been imputed because of communication loss. The idea was that degraded data quality might coincide with severe weather events or grid faults that affect demand. The feature produced no consistent improvement and increased preprocessing complexity, so we dropped it.

We also tested electricity spot price, including intraday and day-ahead MIBEL spot prices. The premise was that large consumers might reduce industrial load when prices rise. For residential feeders, the correlation was essentially zero. Results on mixed feeders were small and inconsistent, so the input remains optional for feeders with known large-consumer price-sensitive load and is off by default.

A school calendar binary feature was tested because school days and school holidays might produce different patterns on feeders serving residential areas with high family occupancy. Most of its effect was already captured by public holiday and week-of-year features. Its marginal improvement fell below the threshold, while its data would require manual maintenance, so we removed it.

A practical feature selection rule

Every added feature brings more than model complexity. An input without genuine signal can add noise, raise overfitting risk, and make failures harder to audit. We keep a feature only when it improves held-out validation MAPE by at least 0.5 percentage points on average across the feeder set, OR improves P90 coverage rate calibration by meaningful margin. Below that bar, maintaining the feature pipeline is not worth the accuracy gain. This is our practical criterion, not a universal feature list.