[Data Issue]: Data is missing permanantly on ENTSO-E transparency and several European countries

[flacombe]

When did this happen?

Since beginning of 2022 at least, can be older.

What zones are affected?

FR, CH at least, can be wider

What is the problem?

Several European production units miss EIC declaration and/or production data, at countries scale.

For instance in France, we're waiting for Landivisiau gas unit production data (450 MW) to appear in the aggregated data with no due date. Landivisiau was commissioned on March 31, 2022 and we're going round around year clock without its figures. Currently, national aggregation data are not comprehensive and don't match actual production.

Complete notice of the issue on OpenMod initiative forum https://forum.openmod.org/t/eu-transparency-generation-data-partly-missing-how-to-finally-get-543-2013-16-1-enforced/3337

Several issues are opened at ENTSO-E, ACER and local regulators.

ElectricityMap should display a warning at least in France and Switzerland due to these missing data.

[VIKTORVAV99]

If I understand you correctly you mean the data from Actual Generation per Generation Unit?

This should not be a problem as long as the production is reported in Actual Generation per Production Type since that's what Electricity Maps use for production data.

[flacombe]

TSO IT teams told me that missing units, particularly for Landivisiau case don't appear in aggregation data (as per of 543/2013 16.1.B).

So Actual generation per production type is impacted as well.

[VIKTORVAV99]

Huh, well that's unfortunate but not much the we or the Electricity Maps team can do about it. However I think adding a small notice along the lines of: "Data is gathered on a best effort basis from 3rd party sources and may not be 100% complete" to the FAQ or maybe even on the zone details panel would make sense.

@rasmusjeppsen what do you think about this from a UX standpoint?

[flacombe]

On a short term basis, yes it may be good to add a global notice or even a more precise one on relevant zones.

At a wider scope, power data community should be aware of such issues and ask for more consistency tests regarding aggregates. The least operators can do is to provide a list of what the aggregate is actually composed of, which is never done currently.

[pierresegonne]

Hey @flacombe do you have any idea whether that type of information is automatically retrievable? We're only a small team and we don't have capacity to follow all of these updates to add / remove notices from the UI. So if there is a data stream for this we can think of building a way to automatically add a disclaimer to the zones, otherwise, we'll have to resort to something along the lines of what @VIKTORVAV99 was suggesting

[flacombe]

Hi @pierresegonne

I'm currently advocating at ENTSO-E and ACER to automatize consistency checks and derivate a dataset of missing plants. But it's hard work to convince them (feel free to encourage them as well)

Assets inventories aren't reliable neither, there are missing plants from EIC and it needs to be addressed separately first of all. Those inventories will then be used to seek for missing production figures. In Landivisiau case, it works: the production unit is found in the EIC inventory (17W000001023704Q) but misses production figures so a warning has to be raised.

Consistency is a common matter, ElectricityMap should challenge the data it retrieves this way.

[flacombe]

Hello

Things improve slowly in France on this particular topic. TSO put things up to send Landivisiau unit data to ENTSO-E. We now have visibility on this ~500 MW fossil gas production unit. https://twitter.com/InfosReseaux/status/1599405164208656385

Saint-Nazaire unit is still missing despite it has reached its maximal power a few weeks ago.

It's an issue that we don't observe any 500 MW change in France's gas aggregated data due to Landivisiau integration. It reinforces doubts I have on the aggregation actual contents.

Wait for the next episode, best regards.

[VIKTORVAV99]

After digging into the HTML a bit I can see that it says when that page was last updated but there seems to be no way to tie this to when the data was updated and I can't see that they have set a specific publishing time or delays anywhere.

So as it stands now I don't think we can use this particular source because we have no way of verifying the data timestamp itself. This means might show things on a vastly different time period than it actually represents and I don't think we want to take that risk.

[flacombe]

I don't get what html you're talking about, ENTSO-E transparency?

You are not supposed to change your source, usually provided by TSO or ENTSO-E itself. This issue is a warning about the aggregate quality. Currently, I'm not sure missing units are included in those aggregates.

It is highly suspicious that TSO IT provide a solution to get 500 MW missing unit with no impact on national aggregate, aren't you?

[VIKTORVAV99]

Woops I commented on the wrong issue 😅 My bad, that's what I get for having multiple tabs open with the issues I'm tracking...

EDIT: I'll mark these 3 comments as off-topic tomorrow when I get back to my computer to keep the issue focused on the right thing.

Sorry for the confusion.