bmreports is down?

[corradio]

I see this error on the backend (https://kibana.electricitymap.org/app/kibana#/discover/1710fdd0-2460-11e8-a779-9d01de8d7a71?_g=(refreshInterval:(display:Off,pause:!f,value:0),time:(from:now-1h,mode:quick,to:now))&_a=(columns:!(level,extra.key,message),filters:!(('$state':(store:appState),exists:(field:level),meta:(alias:!n,disabled:!f,index:'93e631f0-245f-11e8-a779-9d01de8d7a71',key:level,negate:!f,type:exists,value:exists)),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'93e631f0-245f-11e8-a779-9d01de8d7a71',key:level,negate:!t,params:(query:INFO,type:phrase),type:phrase,value:INFO),query:(match:(level:(query:INFO,type:phrase))))),index:'93e631f0-245f-11e8-a779-9d01de8d7a71',interval:auto,query:(language:lucene,query:''),sort:!('@timestamp',desc)))

Could it be that we're exceeding some some of threshold? Can anyone investigate?

HTTPSConnectionPool(host='api.bmreports.com', port=443): Max retries exceeded with url: /BMRS/B1620/v1?SettlementDate=2018-06-28&Period=%2A&ServiceType=csv&APIKey=xdz1cxjwbpjruec (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f24a261c198>: Failed to establish a new connection: [Errno 111] Connection refused',))

cc @lorrieq

[lorrieq]

Hey @corradio, their system had a scheduled outage last night. Back online as of 9:15.

Any gap in data can be fixed because the parser accepts a target_datetime. What I don't understand right now is why the map shows a big and seemingly unfeasible change in values over the small gap.

[corradio]

ok great! Thanks for investigating. The gap in data is indeed disturbing. I expect the bias to be removed as they backfill the data.

On Thu, Jun 28, 2018 at 1:31 PM Laurence Quinn notifications@github.com wrote:

Hey @corradio https://github.com/corradio, their system had a scheduled outage https://www.elexon.co.uk/wp-content/uploads/2018/06/EL02838.pdf last night. Back online as of 9:15.

Any gap in data can be fixed because the parser accepts a target_datetime. What I don't understand right now is why the map shows a big jump in data over the small gap. [image: image] https://user-images.githubusercontent.com/6525086/42031635-e5355174-7ace-11e8-8799-f93b74e38cce.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tmrowco/electricitymap/issues/1481#issuecomment-401004104, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlEKCseGHHa2tLgBDBiDSGZRZyNn28mks5uBL51gaJpZM4U60Pa .

[jarek]

That jump is because EM assumes the last data point is still valid for 2 hours (or maybe 3 hours?) if no subsequent datapoints are available. That's then drawn as a straight line.

[corradio]

Ah yes very possible! Good catch

On Thu, Jun 28, 2018 at 8:04 PM jarek notifications@github.com wrote:

That jump is because EM assumes the last data point is still valid for 2 hours (or maybe 3 hours?) if no subsequent datapoints are available. That's then drawn as a straight line.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tmrowco/electricitymap/issues/1481#issuecomment-401123055, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlEKO3Xt35ChwD4akor2UqjVXkKw3v4ks5uBRqzgaJpZM4U60Pa .