CO2 Intensity Forecast

Forecast the CO2 footprint of grid power and time heavy consumers to periods with lots of renewable energy

Table of Content

What problem are we trying to solve?

We install PV rooftop systems to have our own, green energy. We may even add a battery, to extend the sunshine into the night. We may charge electric vehicles and run heat pumps from this. Great!

But then comes winter, cloudy days and weeks … and all we have is dirty old grid power. So we drive our electric vehicles with coal? Really? There is an inspiring start-up Electricity Maps, who tries to bring reason to this madness. It starts with understanding the CO2 intensity of grid electricity, which they calculate for many countries and areas. They also have some excellent thinking in their blog section.

Ideas which are discussed there and elsewhere:

Bottom line - we’d like to look at data for insight. This section tries to help on that:

These are the kind of questions we want to answer with the Entso-E forecast provider in PVForecast.

Under the hood - how do we get there?

The ugly first: The Entro-E Transparency Platform only covers the European grid. So, so we are limited to this region. But in Europe, we are fine … Technically, we can run on about 63 countries and regions listed. But extensive validation has only been done for Germany. (Users are encouraged to publish other validation results in the Github discussion section).

CO2 intensity calculations

The Electricity Maps Wiki describes in detail the methodology of CO2 footprint calculations and emission factors per power source and region. The Transparency Platform Actual Generation per Production Type provides actual data on energy consumption. So, all we need is to multiply emission factors with energy produced, and we get a CO2 footprint.

The following shows the CO2 intensity for Germany in January 2023: Red is as calculated in PVForecast, and yellow is provided from Electricity Maps CO2signal. We can see that we have a pretty good match between the ‘gold standard’ CO2signal and the calculations performed by PVForecast.

Next, we need to forecast CO2 intensity. Luckily, the Transparency Platform provides Generation Forecasts for Wind and Solar as well as overall Generation Forecasts. These forecasts are provided at 18:00 Brussels time for the day ahead and at 8:00 Brussels for the current day. This can be used to calculate a percentage of non-renewable energy as

1 - renewableGeneration/totalGeneration

It turns out that this percentage correlates very well with the CO2 footprint generated: The solid line corresponds to the actual CO2 intensity as explained above, whereas the dotted line is the forecasted footprint at the time of writing this article. It appears that tomorrow is a sunny day across Germany with lots of solar energy at noon, causing a welcome dip in CO2 pollution.

Whilst we see, that on a short period of a few days, the said percentage and the CO2 footprint correlate well, this cannot be assumed for a long period (eg. years): Grid properties change, power plants go in maintenance, gas/coal mix changes, nuclear retires, etc. Hence, the correlation needs constant re-tuning: correlation coefficients are continuously calculated based on the last few days of actual vs. forecasted data. The resulting overall prediction is then stable over a long period, below for 2020 - 2022 (each dot corresponds to a 15min interval)

The number of days to establish the correlation should not be too short (noise reduction), but also not too long (as it won’t be responsive to changes of grid properties). It was found that 7 days are an optimum, but could be extended to ~21 days without much loss in accuracy. The following shows forecast errors for three different regression intervals (7, 14 and 21 days):

The correlation period can be adapted with modelDays in the config file, defaulting to 7 days.

Load and auction price

Load forecasts and Auction Prices are readily available on the Transparency Platform

It is obvious that load is lower over night and on weekends. Hence, everything else the same, big consumers should focus on night hours for the most grid-friendly behavior - ie., in order to not overload the grid. As can already be seen, market prices (gray line) correlate somewhat with the CO2 footprint. That’s not too surprising, as renewable energies are the cheapest today.

To illustrate this further, we can look at the below graph: for every hour of the time frame 2020 - 2022, the deviation of the market price to the 40-day moving average is plotted in y. On x we plot the percentage of renewable energies:

The use of a deviation from a 40-day moving average filters out the impact of the energy crises on prices, due to the war: average prices increased from 40Eur/MWh in 2020 to more than 600Eur/MWh in Autumn 2022 and then started dropping agin. After filtering these longer-term effects, a correlation with the renewable energy percentage becomes visible. With >60% renewables, electricity prices can even drop into the negative, as they then start competing with sources which cannot be switched on and off fast enough, resulting in an oversupply.

Supported Countries and Regions

The CO2signal is available for the countries and regions shown on the Electricity Maps map. Unfortunatly, the free API does not provide forward looking data.

The Transparency Platform has data on Europe. We interface to it with entsoe-py which lists 63 countries and control regions. But not all of these regions support all data types. EntsoE_Zones.pdf shows a snapshot of what was available in late December 2022:

The Entso-E module calculates following columns (if corresponding data is available):

Colum Name Column
co2 CO2 intensity, calculated from generated actual (g/kWh])
co2_forecast (*) forecasted CO2 intensity forecast, based on the most recent forecast (DayAhead or Intraday)
pctGenerated_DayAhead 1 - renewableGeneration/totalGeneration, based on DayAhead forecast
pctGenerated_Intraday same, but based on Intraday forecast for renewables
pctLoad_DayAhead 1 - renewableGeneration/totalLoad, based on DayAhead forecast
pctLoad_Intraday same, but based on Intraday forecast for renewables
price bidding prices (in Eur/MWh)

(*) if the Python library scipy is not installed, this is the most recent pctGenerated_xxx figure. It will give the correct trend (ie., low is good, high not so much), but not absolute numbers.

Sanitizing data for a zone

Technically, this should all run - otherwise we have a bug in the code and raise an Issue. But it’s a different question whether the data makes sense. Substantial sanitizing has been done for Germany, but not for other countries and control regions.

Control Regions vs. Country

Germany is subdivided into four control regions. Bidding prices are identical and combined with LU in the zone DE_LU. But generation and load data are available separately. It may then not surprise very much that Amprion has consistently a higher CO2 footprint than Germany as a total: The region doesn’t border to the sea, hence has no off-shore wind power. But it does have the huge coal power plants of Garzweiler. Green shows CO2 footprint for the Amprion region, red for DE:

We assume that data from the Transparency Platform is accurate. However:

Hence, if we venture in zones outside Germany, we need sanitize the data.

Sanitizing CO2 Intensity Data

This can be done in two steps:

  1. compare production and consumption CO2 intensity on Electricity Maps App for a while - are there huge differences? Likely, for small countries impact of import/export will be bigger than for large countries. Here we work only with production data - does this achieve the desired goal?

  2. run CO2signal and Entso-E in parallel for a while and compare columns co2signal_<zone>.carbonIntensity and entsoe_<zone>.co2: Are the differences tolerable?

Comparisons for Germany shows that CO2signal sometimes is off for a few hours, but never for long. This is because they calculate CO2 hourly, but don’t correct backwards. Hence, in case grid operators deliver data delayed we won’t see this corrected in the free API. PVForecast on the other hand always updates with a backward looking time window of 24h. The SQLite storage model could be used to check when corrections were made, since it stores data of each download separately.

Sanitizing the forward looking fit for CO2 forecast

We could let PVForecast run for a while to see how well the fit is going to work for other zones, other than DE and related. This is kind of boring. However, the Transparency Portal allows to download up to one year worth of data. We can do this as follows:

[Entso-E]
    api_key   = <api_from_Entso-E>
    zones     = DE                   # comma separated list of zones to be analyzedDE  #, FR
    keepRaw   = 1                    # keep all downloaded raw data columns
    start     = 2023-01-01T23:00Z
    end       = 2023-02-18T23:00Z
    loop      = 0
    storeCSV  = 1

This will storeCSV all data (including all columns downloaded, due to keepRaw = 1) between start and end. This can then be analyzed in Excel. Note that the generated table also contains co2_forecast. However, this number does not use rolling correlation coefficients of the respective last few days.

Once we have sanitized the proper working of the forecast model, we can backload the database - for the last week, month or year. To do this, we set loop = 1 and let it run: It will download each day separately, calculate the rolling correlation coefficients and store the result in the database. This option doesn’t make sense if not at least one of storeInflux or storeDB is enabled. storeCSV is disabled for looping.

A possible strategy to use this data

Based on data described here, and in the PV Output Power Forecast, an energy use strategy with the following priorities might make sense (not considering any financial aspects)

  1. Conserve energy as much as possible - there is no such thing as free solar energy; through grid feed-in somebody else might benefit
  2. Noon peeks on sunny days: most likely, there is a surplus of renewable energies during these times, so running hot water preparation during this period is a good idea
  3. Energy created locally - be it directly from PV system or stored in home battery: This limits load on the grid and is essentially CO2 free
  4. Use energy when local control region has low CO2 foot-print - plan EV charging during such times, if not possible from own rooftop PV
  5. Charge EV during night hours, when grid load is low - preferably during times when also country-wide CO2 foot-print is low

Further references