search

electricitymaps / electricitymaps-contrib

A real-time visualisation of the CO2 emissions of electricity consumption
https://app.electricitymaps.com
GNU Affero General Public License v3.0
3.46k stars 923 forks source link

[Data Issue]: Erroneous data for Western Australia 22 and 23 December (CET) #6287

Open q-- opened 8 months ago

q-- commented 8 months ago

When did this happen?

Dec. 22 and 23 (CET)

What zones are affected?

AU-WA

What is the problem?

Production modes seem to disappear at some time points:

Data error for Western Australia - screenshot of part of the zone panel in ElectrictyMaps

q-- commented 8 months ago

By the way, is it useful if I make an issue for errors like this that are probably due to the data source, or do your automated systems catch this?

VIKTORVAV99 commented 8 months ago

By the way, is it useful if I make an issue for errors like this that are probably due to the data source, or do your automated systems catch this?

We do have some automated systems to correct it (but not really flag it yet) that run on a schedule.

We are creating an automated system to flag data though but it has not been deployed yet (hopefully in January though).

So until then these are great and help me keep tabs on them (and manually run our pipelines if needed).

VIKTORVAV99 commented 8 months ago

Looks like the source data is also the cause here, which is unfortunate.

Source: https://opennem.org.au/energy/wem/?range=30d&interval=1d&view=discrete-time

Most of our team are on holidays now but if this persists we will manually flag this data so our estimations can take over instead, not ideal but should be better than the current data.

VIKTORVAV99 commented 8 months ago

Looks like we also lost the ability to backfill data. I have opened a issue for it at https://github.com/opennem/opennem/issues/349, hopefully we will be able to restore this ability as soon as possible and refetch all the affected data once it has been fixed at the source.

This also means those automated systems I talked about don't really work as expected unfortunately.