GB-NIR is down

[corradio]

The GB-NIR parser is down:

Traceback (most recent call last):
  File "test_parser.py", line 132, in <module>
    print(test_parser())
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/.venv/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/.venv/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/.venv/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/.venv/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "test_parser.py", line 65, in test_parser
    *args, target_datetime=target_datetime, logger=logging.getLogger(__name__)
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/parsers/GB_NIR.py", line 214, in fetch_production
    production_data = get_data(production_url, target_datetime)
  File "/Users/oliviercorradi/github/tmrowco/electricitymap/src/electricitymap/contrib/parsers/GB_NIR.py", line 45, in get_data
    viewstategenerator = soup.find("input", attrs={'id': '__VIEWSTATEGENERATOR'})['value']
TypeError: 'NoneType' object is not subscriptable

It turns out the website has changed. I found out it uses the URL http://smartgriddashboard.eirgrid.com/DashboardService.svc/data?area=fuelmix&region=NI&datefrom=2020-01-01&dateto=2020-03-03, but changing the date has no effect. Unfortunately ENTSOE data is not good enough (there's a lot of missing data)

Here are suggested next steps:

• Only collect real-time data using https://github.com/tmrowco/electricitymap-contrib/pull/2916
• Contact data provider and see if we can get access to both live and historical data

[FelixDQ]

I have written an email to EIRGRID to see if we can get all the data needed. I will update when they respond

[corradio]

cc @Kongkille

[FelixDQ]

No response yet. We've tried reaching out to a couple more people to see if we can get the data.

The alternative would be to switch to ENTSOE. Unfortunately, the data from ENTSOE seems to be lagging a bit (~24 hours), so let's see if we can get real-time data before. I also reached out to ENTSOE to ask if they expect to be able to provide real-time data at some point.

[lorrieq]

Hey, I'm not familiar with the overall state of this issue or #2916 but a quick fix for the exchange would be to use _ELEXON.fetchexchange for the GB->GB-NIR flow. It should be as simple as changing the exchanges config.

[VIKTORVAV99]

@corradio @FelixDQ can we close this issue or is it still relevant?

[corradio]

I'll let @FelixDQ comment as I don't have context anymore.