search

electricitymaps / electricitymaps-contrib

The open source repository for Electricity Maps App and data parsers that enables a real-time visualisation of the CO2 emissions of electricity consumption
https://app.electricitymaps.com
GNU Affero General Public License v3.0
3.62k stars 952 forks source link

Add Canada #166

Closed corradio closed 5 years ago

corradio commented 7 years ago

Ontario: http://live.gridwatch.ca/home-page.html

GuillaumeHo commented 7 years ago

In Canada, the provinces have the control of their energy policy, hence the regulation of their electric power. 2 Provinces, Ontario and Alberta switched from regulated prices to an energy market. Here are the provinces and the pages or API I found to follow the production by type of energy and the exchanges with other provinces.

On a federal (=national level) the National Energy Board issues a report every year (here the data for 2015). Could be used to have static data on energy type https://www.neb-one.gc.ca/nrg/sttstc/lctrct/rprt/2016cndrnwblpwr/2016cndrnwblpwr-eng.pdf

Canada is also good at open data, so in the future if we want maps or positions of plants, electric connexions with the US etc. http://open.canada.ca/en/open-data

Three Federal territories

corradio commented 7 years ago

Sounds terrific! I'll work on preparing the map itself so we have the right topologies.

jarek commented 7 years ago

Hi, I saw that some Canadian data started appearing in Country Ranking and found the region=northamerica page. I would like to contribute a bit to this. Is anyone else working on any Canadian data right now? The Alberta data linked to by Guillaume looks like it'll be easy to integrate.

corradio commented 7 years ago

Go ahead @jarek - would be awesome to have your contribution! Let me know what you need, or ask us on Slack!

corradio commented 7 years ago

Hey @jarek ? Are you on Slack? (http://slack.tmrow.co). We're preparing a big launch soon and it would be nice if AB would be there!

jarek commented 7 years ago

Sorry @corradio, I ended up having no time so far :( I see mlucchini has done Alberta, congrats! I might try to integrate some other provinces.

I notice there are no capacities for Canada - should they be named with underscore (like NZ-NZN in zones.json) or with dash (like CA_ON.py in parsers)?

corradio commented 7 years ago

Hi @jarek. No worries. Glad we did no do double work :) Happy to get some capacities :) They should be named with a dash (like in zones.json).

On Tue, Apr 25, 2017 at 8:13 PM, jarek notifications@github.com wrote:

Sorry @corradio https://github.com/corradio, I ended up having no time so far :( I see mlucchini has done Alberta, congrats! I might try to integrate some other provinces.

I notice there are no capacities for Canada - should they be named with underscore (like NZ-NZN in zones.json) or with dash (like CA_ON.py in parsers)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tmrowco/electricitymap/issues/166#issuecomment-297118475, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlEKByLOPcskTzKLRWgOBw8qF_pwcETks5rzjezgaJpZM4LMJC- .

jarek commented 7 years ago

There are no links to current generation numbers that I saw, but the "Provinces and Territories" sections of http://peoplepowerplanet.ca/ has a decent overview of electricity/energy sector per province, and might help people trying to figure out what numbers are what or what is roughly correct.

jarek commented 7 years ago

I have also been working off Statistics Canada historical data, which is pretty good. Follow links from http://www5.statcan.gc.ca/cansim/a33?lang=eng&spMode=tables&themeID=1744&pattern=&stByVal=2&RT=TABLE, particularly "Table 127-0002: Electric power generation, by class of electricity producer; monthly (megawatt hour)" and "Table 127-0008: Supply and disposition of electric power, electric utilities and industry; annual"

It doesn't have any real-time data, and the categorization of fossil fuels doesn't line up with electricitymap's (it has e.g. "Conventional steam turbine" and "Combustion turbine" rather than actual fuels), but particularly 127-0002 is pretty good in understanding what each province's supply looks like. Data is up to 2015, in most cases annual and in few cases monthly.

jarek commented 7 years ago

Saving research notes for if anyone else is looking at Ontario:

Per http://www.ieso.ca/learn/ontario-supply-mix/ontario-energy-capacity and http://www.ieso.ca/-/media/files/ieso/document-library/contracted-electricity-supply/progress-report-contracted-supply-q12017.pdf?la=en there is about 1.9 GW of distributed solar (this would include residential rooftop installations) and about 500 MW of distributed wind generation in Ontario. These are not included in the IESO data because that only includes what goes over high-voltage parts of the grid. There appears to be no real-time data or estimate for this production currently, though http://live.gridwatch.ca/total-capacity.html says IESO is working on embedded generation reporting. Per Table 3 in the PDF, distributed generation is about 11.5% of the total capacity, and it's mostly solar.

As there is no real-time data or official estimates, we would have to make our own estimates as to how much the distributed generation is producing. To keep things simple, we're opting against custom estimates for now, and only show what IESO is showing.

jarek commented 7 years ago

We would love to get real-time generation breakdown for Quebec, British Columbia, Manitoba, Saskatchewan, New Brunswick, Newfoundland and Labrador, Nunavut, and Northwest Territories. If anyone knows of sources, or can ask for one to be opened to the public, please contact us!

HansHyde commented 6 years ago

@jarek, @corradio

Updates for Quebec

Possible places to look for realtime data

  1. App? - | https://itunes.apple.com/ca/app/hydro-quebec/id1045080947?l=fr&ls=1&mt=8
  2. HQ Reliability Coordinator - | http://www.hydroquebec.com/reliability-coordinator/
  3. Contact the HQ RC - | https://www.hydroquebec.com/sefco2016/en/reliability-standards.html

Updated capacities - all others not owned by HQ, have contracted feed-in terms into HQ

No. 3 may be worth a shot for @brunolajoie to contact, show off Electricitymap.org and see if we can get access to realtime data. The RC is to the best of my knowledge the system operator (ISO)

Additional info for exchanges to keep in mind -

  1. 15 with Ontario, New Brunswick and the U.S. Northeast.
  2. 6,025 MW of import capacity
  3. 7,974 MW of export capacity

I also added the nuclear plant, although it was closed Dec 28th 2012, with a "comment" linking to more information. Nuclear could be taken off the Quebec table, but maybe reactors recently closed is knowledge worth having - as is possibly "recently closed coal capacity"??? Food for thought...

updates passed in local mockserver, tried to link commits to this issue

jarek commented 6 years ago

@HansHyde - I would keep Churchill Falls in Newfoundland and Labrador for now. Until we have real-time data for either, changes either way are academic. For what it's worth, the aggregate production data from Statistics Canada has Churchill Falls output listed in NL.

I would also not add the QC nuclear power plant. It's not coming back. Similarly Ontario shows 0 coal capacity for "recently closed".

I'm wondering if the opening of Maritime Link is an opportunity to press for real-time data on its operation. Or if they'll rather keep it a secret, as the associated hydro project is beleaguered enough as it is...

corradio commented 6 years ago

@jarek you should probably review https://github.com/tmrowco/electricitymap/pull/942 if you have additional info

HansHyde commented 6 years ago

Hi @jarek No. 1 - I humbly disagree as a page deeper than the source you pulled original capacities from specifies the terms of Churchill Falls - see image (data I compiled from the source data)

hq - gen capacites contracted capacities

Pulling from Statistics Canada is higher level (almost qualitative data), and would be similar to us getting data from EIA in the US, when we can go to the ISO/RTOs for higher level quantitative data. Good for baseline, but open to further refinement & accuracy.

No. 2 - I agree with you. For a later date, it might be nice to include closed power plants by type & installed capacity within the past 10 years. In a "war on carbon emissions" perspective - nuclear closures could be seen as a negative, coal closures a positive, etc.

No. 3 - Sorry, not looked closely at the Maritime link (trying to sort US regions). Points re:

jarek commented 6 years ago

I also asked about assuming co2eq of imports from a zone with unknown imports in #563, there is no easy way to do this right now in the system.

jarek commented 6 years ago

I will close this issue as currently all known real-time information is integrated in Electricity Map. Please reopen if more information is identified or becomes available.

jarek commented 6 years ago

The PEI data source gives only the current data. However, I have just ran across https://energy.reinvented.net/, which is operated by Peter Rukavina (who runs the ruk.ca website linked in the second comment). It contains a scraped CSV archive of the data, every 15 minutes, going back to November 2012. If we are ever interested in getting archive data for PEI, it might be the ticket.

A new API endpoint for PEI data is also named at https://ruk.ca/content/new-api-endpoint-pei-wind, though the old one still seems to work for now.

Minor data point, the interchange between PEI and NB now has capacity of 360 MW per https://www.cbc.ca/news/canada/prince-edward-island/pei-electric-underwater-cable-northumberland-strait-1.4267315 and other similar news stories.

reinvented commented 6 years ago

I was told that the endpoint for Prince Edward Island, at http://www.gov.pe.ca/windenergy/chart-values.php would be deprecated on Oct. 31, 2018. That hasn't happened, but I believe that is still the plan.

Here's the technical information I received about the replacement endpoint:

Wind Energy Endpoint POST https://wdf.princeedwardisland.ca/prod/workflow

Request Header Content-Type : application/json

Request Body

{
"featureName": "WindEnergy"
}

Response Body

Note: The decimal value will need to be parsed from the "header" string. At first glance you may think to use actualValue but this is a truncated value for display on the gauge. We are aware of this and will be changing in a future release.

{
"components": [
 {
  "type": 7,
  "data": {
   "actualValue": 198,
   "footer": "",
   "maxValue": 300,
   "header": "Total On-Island Load: 198.46 MW",
   "color": "#008570"
  }
 },
 {
  "type": 7,
  "data": {
   "actualValue": 92,
   "footer": "",
   "maxValue": 200,
   "header": "Wind Power Used On Island: 92.41 MW",
   "color": "#008570"
  }
 },
 {
  "type": 7,
  "data": {
   "actualValue": 188,
   "footer": "",
   "maxValue": 300,
   "header": "Total On-Island Wind Generation: 188.63 MW",
   "color": "#008570"
  }
 },
 {
  "type": 7,
  "data": {
   "actualValue": 96,
   "footer": "",
   "maxValue": 200,
   "header": "Wind Power Exported Off Island: 96.22 MW",
   "color": "#008570"
  }
 },
 {
  "type": 7,
  "data": {
   "actualValue": 0,
   "footer": "",
   "maxValue": 200,
   "header": "Total On-Island Fossil Fuel Generation: 0.00 MW",
   "color": "#008570"
  }
 },
 {
  "type": 5,
  "data": {
   "text": "Last updated October 24, 2018 09:59 AM"
  }
 }
]
}

Sample cURL command

curl -X POST -H 'Content-Type: application/json' -d '{"featureName":"WindEnergy"}' -v -i 'https://wdf.princeedwardisland.ca/prod/workflow'

jarek commented 5 years ago

new PEI endpoint implemented in #1791

jasonrwang commented 3 years ago

FYI, there is an emerging project from Natural Resources Canada (a federal ministry) to obtain and provide real-time electricity data in a single portal. If and when this project continues, I'd be happy to help dig up more information about its implementation to tie in here and fill in data for the missing provinces and territories, and maybe supersede the existing province-specific parsers.

pbubik commented 3 years ago

Why is export from Manitoba to Ontario showing intesity of 461g/kWh? Manitoba is largely hydro and wind, there is no way they have 461g/kWh

madsnedergaard commented 3 years ago

Why is export from Manitoba to Ontario showing intesity of 461g/kWh? Manitoba is largely hydro and wind, there is no way they have 461g/kWh

The Ontario parser fetches imports - so we know that Ontario is importing from Manitoba, but since we don't know have a parser for Manitoba or know the production breakdown, we fallback to a world average intensity. This can be improved by updating the fallback mix in config/co2eq_parameters.json, so if you have a source for the production breakdown we'd be happy to see a PR that updates it :)

See example for British Columbia here:

https://github.com/tmrowco/electricitymap-contrib/blob/2009ad04be124eec67651408b2d9292dd5bb36ba/config/co2eq_parameters.json#L1209-L1218

pbubik commented 3 years ago

Ah, I C. Manitoba is cca 97% hydro, but I don't code, so I can't fix the error. @madsnedergaard Could you do it? Manitoba power mix is here: https://en.wikipedia.org/wiki/List_of_generating_stations_in_Manitoba

I just e-mailed Manitoba Hydro if they would be willing to share real-time data, I wouldn't hold my breath though. :)

pbubik commented 3 years ago

Also, a good source of GHG intensity for Manitoba is here: https://www.hydro.mb.ca/environment/greenhouse_gas/

"Nearly all our grid-connected electricity is generated at hydroelectric generating stations. We also operate 2 natural gas generating stations as back-up for our hydroelectric system.

Annual GHG emission intensity factor from our electricity generation 2020: 0.44 tonnes CO2e/GWh; 2019: 0.88 tonnes CO2e/GWh; 2018: 0.43 tonnes CO2e/GWh; 2017: 1.27 tonnes CO2e/GWh; 2016: 1.15 tonnes CO2e/GWh; The global perspective considers changes made to incremental electricity consumption, such as through energy efficiency. We evaluate these changes based on the regional GHG effects. We currently use a factor of 750 tonnes CO2e/GWh."

madsnedergaard commented 3 years ago

The data in the Wikipedia entry is from a report published in 2015. I'm struggling with finding more recent data 🤔

So far using https://www.hydro.mb.ca/corporate/facilities/generating_stations/ might be the best option, although that one also seems outdated (the new Keeyask generating station has started producing power, but isn't in there)

madsnedergaard commented 3 years ago

FYI @pbubik, the change is now live :)

Screenshot 2021-07-30 at 09 18 29

jarek commented 2 years ago

Just to let people know, Statcan table 25-10-0015-01 (formerly table 127-0002) now has monthly data for electricity production in all provinces and territories delayed only by a few months, currently up to May 2022.

For most (all?) regions it merges all "combustible fuels" (coal, gas, oil, maybe biomass?) into one category, but has good data for hydro, wind, solar, and nuclear.

For example for British Columbia: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=2510001501&pickMembers%5B0%5D=1.11&pickMembers%5B1%5D=2.2&cubeTimeFrame.startMonth=04&cubeTimeFrame.startYear=2020&cubeTimeFrame.endMonth=11&cubeTimeFrame.endYear=2022&referencePeriods=20200401%2C20221101

jarek commented 2 years ago

As posted by @b-neufeld in https://github.com/electricitymaps/electricitymaps-contrib/issues/3633, Saskatchewan now has a dashboard that seems to be updated daily: https://www.saskpower.com/Our-Power-Future/Our-Electricity/Electrical-System/Where-Your-Power-Comes-From

JSON source is https://www.saskpower.com/ignitionapi/PowerUseDashboard/GetPowerUseDashboardData

Right now (morning of Sept 29), it gives data with descriptive date "Sep 27, 2022" a timestamp 1664344799 which is 2022-09-27 23:59:59 - so dated at end of day, with a delay of at least 1 day.

The page displays generation in "MW" but doesn't give a time when this generation happened. Page says "View our daily snapshot below that’s an average of a 24-hour period." which is a bit of a strange way of doing it. What does this represent - actual generation in MWh over 24 hours added up and divided by 24 hours (not suitable for Electricity Maps)? Peak generation at some point during the day (but not timestamped)? Generation at midnight when the data is timestamped? It gives total generation as 2886 MW which roughly lines up with previously found "system load" figure in https://www.saskpower.com/Our-Power-Future/Our-Electricity/Electrical-System/System-Map (currently 2617 MW, at 7am local time).

Without more frequent updates, this isn't super useful for Electricity Maps, but might be worth keeping an eye to see if they feed in more frequent data. Fingers crossed for hourly sometime!

b-neufeld commented 2 years ago

@jarek some thoughts from a SK local...

  1. I think your first hunch is right, it's actual generation in MWh over 24h, divided by 24. This is silly and even if it doesn't work for Electricity Maps I'm happy because until now we had 1 data point per year in the annual report :)
  2. The Where-Your-Power-Comes-From page doesn't give peak generation during the day, but we could grab that from the System-Map page. The total system load value updates every 5 minutes.
  3. the Generation/Demand/Exports parameters are confusing and there's another one called Supply hidden in the json, maybe accounts for line losses? I've started historizing all this locally to see what means what, will update this thread if I catch any useful changes.
  4. The datestamp is also confusing... once I have a few more days of history I should be able to verify what 24h period the daily stats actually are referring to.