Ireland data is missing

[corradio]

ENTSOE tells us that the data provider has some issues and are not communicating data. I'll try to keep this thread updated as we go along. If anyone can help in pushing, feel free to do so.

[systemcatch]

I sent an email to EirGrid asking about getting real time type breakdown for generation data. Received this in response which is quite positive.

Good afternoon Chris and thank you for your suggestions re the dashboard.

I have passed them on to my colleagues who have responsibility for this application to consider.

In the meantime, if generation by fuel type is of interest to you immediately, and if you are not aware of this already, could I point you in the direction of SEM-O http://www.sem-o.com/Pages/default.aspx

You can access generation data on their Dynamic Reporting section http://www.sem-o.com/MarketData/pages/default.aspx?ReturnUrl=%2fmarketdata%2fPages%2fdynamicreports.aspx (a quick sign in process required initially).

To find out the fuel type of the units in the market you can access information from this page: http://www.sem-o.com/JoiningTheMarket/registerageneratorunit/Pages/generator.aspx - "useful links List of Registered Units (.pdf)
List of Registered Units (.xls)

Kind regards,

Mary Doyle Customer Relations.

[thomasgibon]

For what it's worth, it seems EirGrid actually has live data, updated every 15 minutes, here: http://smartgriddashboard.eirgrid.com/ Except the electricity source breakdown is only for the last 24 hours (and updated every 24 hours I think).

I started to write a parser for it, yet to be tested: https://github.com/thomasgibon/electricitymap/blob/master/parsers/IE.py

[systemcatch]

Hey @thomasgibon we would like to use that source but without a better live breakdown by type it's not going to work. I've been in contact with EirGrid (see earlier in issue) about getting this changed but so far they have not altered anything.

What would be useful for now is implementing the exchanges that Ireland has with other countries along with the electricity price. You might have seen that we're moving over to Python 3 so your parser will need to run using that not Python 2.7. If you need a hand with anything just ping us on slack or post here. Glad to have you helping out.

[alixunderplatz]

@thomasgibon Nice! I came across that dashboard, too.

I was wondering, how to deal with the fact that generation breakdown is the average for the entire last 24 hours. I think a daily average of the types will not be accepted for the electricitymap, since the data should be as fresh as possible. I'm not good at reading code, so may I ask you, how you associate the recent generation with the fuel types?

I think a temporary workaround is to split generation to "wind" (which is available seperately) and "unknown". As far as I remember from the past, a coal and gas-mix was present most of the time, with gas peaking at low wind speeds only.

@corradio @systemcatch Something very different and easier we could try is leaving the generation blank and use their own provided CO2 intensity for Ireland.

It's available right here: http://smartgriddashboard.eirgrid.com/#roi/co2 it's also seperately available for Northern Ireland and the whole island of Ireland

They describe the data as follows:

CO2 Emissions The rate of CO2 emissions is calculated in real time by using the generators MW output, the individual heat rate curves for each power station and the calorific values for each type of fuel used. The heat rate curves are used to determine the efficiency at which a generator burns fuel at any given time. The fuel calorific values are then used to calculate the rate of CO₂ emissions for the fuel being burned by the generator.

I think the published intensity is really precise, based on real power plant efficiencies and is also updated regularly. Are there any concerns about using their CO2-intensity?

This would bring Ireland back almost instantly. A split of the generation to "wind" and "unknown" could still be performed, while we override our emission factors with their intensity.

[corradio]

Unfortunately that will be quite complicated (i.e. to integrate a specific co2 intensity, calculated using different assumptions). Unless we have the proper breakdown we can only pressure the data providers to make sure they feed ENTSOE properly.

[thomasgibon]

Thanks for your answers!

@systemcatch Strange, the code is Python3, what made you think it was 2? Yes, I think I'd need help, I'll try to swing by on Slack :)

@alixunderplatz @corradio The issue is indeed how to apply the mix to the live data... The mix is only given a posteriori (after the electricity is generated), so it would be wrong to apply it to the instantaneous generation. I do that in my parser, as a quick fix, but it is obviously wrong indeed. The workaround you suggest (wind/other) is better, and maybe we can give to "other" the CO2 content of the 24-hour mix?

And yes, I have a concern about using EirGrid's CO2-intensity: there is an important difference between the CO2 numbers that EirGrid provides, and the IPCC ones. The former are only direct emissions (based on heating values of the fuels), while the latter are life-cycle based. In that perspective EirGrid would e.g. give a factor 0 to wind or hydro. That would discard EirGrid's values unless indirect CO2 emissions are added separately - which is no easy task. The wisest is probably to wait for better data.

[alixunderplatz]

@thomasgibon I agree, waiting for better generation data seems to be the only real good option. I was just very enthusiastic after seeing the intensity graph.

Right now, there is a pretty good correlation for the Northern Ireland data to the calculations which are used by electricitymap. 240 vs. 243 g/kWh at 17:00 local time. I'll try to watch the deviations in the next few days.

Previously, the data from EirGrid were even showing much higher CO2 intensity than for the IPCC numbers, which in my opinion is always a preferable case. Tonight, emissions were beyond 600 g/kWh according to EirGrid, while electricitymap had something between 250 to 300 g/kWh.

I have no intention to start another debate on the IPCC numbers 😇 - I see it's good to use unified numbers for comparison reasons.

[jarek]

Meanwhile it looks like Ireland data is back in ENTSOE :tada:

As I don't think we got to a point where we can implement another parser based on EirGrid data, should we close off this issue?

[corradio]

I believe so

On Sun 11 Mar 2018 at 22:17, jarek notifications@github.com wrote:

Meanwhile it looks like Ireland data is back in ENTSOE 🎉

As I don't think we got to a point where we can implement another parser based on EirGrid data, should we close off this issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tmrowco/electricitymap/issues/823#issuecomment-372149996, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlEKATqnO3va5UevDQd3yXPiKeX_cshks5tdZRYgaJpZM4QQt2N .