poissonconsulting / fish-passage-22

0 stars 1 forks source link

Real time water temperature data from BC hydrometric stations #13

Open NewGraphEnvironment opened 7 months ago

NewGraphEnvironment commented 7 months ago

Real time water temp data is being collected at 140 hydrometric stations and is accessible from the ECCC web service (easy to grab with tidyhydat::realtime_ws()).

Since I went through the process and need somewhere to keep it I will do a PR to add a read-water-temp-realtime.R to this repo. Totally not offended if you don't want to merge. Just wanted to put it somewhere. The burned results are also added to the dropbox as data/realtime_data.sqlite

Shockingly this temp data is only available for 18 months back from the current date and then is not stored somewhere publicly accessible! I will make a data request from ECCC for everything they have but not holding my breath. Going forward, if we cannot easily get access to this data i propose we build a github action to automatically download all the temp data each month and add it to the sqlite. We could also explore options like adding this data to Living Lakes CKAN database ( https://data.cbwaterhub.ca/ ) to make it accessible to all...

Wouldn't think it would be hard to tie this temp data to discharge from hydat (through sqlites provided handily in tidyhydat) once it gets published by ECCC. Maybe not a great option for this particular fiscal but going forward something to consider

NewGraphEnvironment commented 7 months ago

here is the data request prepped to send to the weaterh.gc.ca inquiries portal ( Contact Us - Environment Canada (weather.gc.ca) ). Thinking we could consider makeing the repo public before sending. Feel free to modify as you see fit too.

I did reach out to a contact that is a "Hydrometrics Specialist" at the Ministry of Environment & Climate Change Strategy and their supervisor indicated that "Temperature isn’t an official data product so they don’t quality control it. They can release data for it, if it’s available, but it’s not available at all sites." and that a data request was likely the best route.

Hi there.  Sorry for all the text. Thank you for your time!

Hoping you could provide access to historical water temperature data BEFORE June 18, 2022 ("2022-06-18 00:00:00 UTC") for all 129 hydrometric stations where it is available in British Columbia for as far back as possible.

We were able to gather the past 18 months of realtime water temperature data up to January 18, 2024 ("2024-01-18 00:00:00 UTC") from the ECCC web service for all 129 hydrometric stations in British Columbia where it is available using this script https://gist.github.com/NewGraphEnvironment/0dbc6f6d184cfeb149501b58d2c91d21. We are using this data as part of an analysis linking location on BC freshwater atlas to temperature, discharge and fish density as part of an evidenced based approach to inform ecosystem restoration investments.  Here is a link to the repository where we are currently working on the analysis ( https://github.com/poissonconsulting/fish-passage-22/ ).  Many of our projects are completed on behalf and in partneship with the non-profit Society for Ecosystem Restoration Northern BC and can be viewed here https://www.newgraphenvironment.com/project/aquatic_restoration_planning/ .  

Guessing you don't need a list of sites where this data is being collected but we will include it in case it is useful. Ideally we would access the data programatically through an API but any which way from Sunday is better than nothing at all so happy to deal with whatever. Feel free to call at 250 777 1518. Sorry for all the text. Thank you!!!

07EA004 07EB002 07EC003 07EC004 07EE007 07EF002 07EF004 07FD002 07FD019 08CD001 08CE001 08CE005 08CG001 08DA005 08DB001 08DB013 08EB003 08EB004 08EB005 08EC001 08EC004 08EC013 08ED001 08ED002 08EE003 08EE004 08EE005 08EE012 08EF005 08EG019 08FA007 08FB006 08FB011 08FF001 08FF002 08FF003 08FF006 08GA047 08GA077 08GB013 08GC008 08GD004 08GD010 08GE002 08GF007 08HA001 08HA002 08HA003 08HA010 08HA011 08HA016 08HA068 08HA069 08HA070 08HA072 08HB014 08HB023 08HB032 08HB034 08HB048 08HD006 08HD035 08HF004 08HF014 08JA017 08JA018 08JB002 08JB003 08JB008 08JC001 08JC002 08JE001 08KB001 08KB003 08KD007 08KH006 08KH020 08LB064 08LB078 08LC039 08LC042 08LE024 08LE027 08LE031 08LF002 08LF027 08LF033 08LF051 08LG010 08LG065 08MA001 08MB007 08MB012 08MC018 08MD013 08ME003 08ME023 08ME030 08MF005 08MF040 08MG022 08MH141 08NB005 08NB021 08ND012 08NE049 08NE074 08NG065 08NH118 08NK002 08NL045 08NM002 08NM050 08NM083 08NM085 08NM143 08NM158 08NM172 08NM200 08NM247 08NN002 08NN003 08NN026 08OA002 08OA003 08OB002 09AA001 10AC005 10DA001
NewGraphEnvironment commented 7 months ago

Guessing this link is only shared with me but we have the data if we want it. Email text below and screenshot of drive directory and example csv. Obviously, we could harvest this temp data and tie to hydometric data farmed from Hydat. A discussion of whether it is worth it (as part of this particular analysis or something separate) also obviously a good idea.

Thank you for your inquiry. I have a collection all our water temperature data prior to 2022-12-13 and can be found here: https://drive.google.com/drive/folders/1laXNGO0cMUCAC_l8IhfmhG1tXdnUE2Jh?usp=drive_link

You will find stations beyond what you have listed below as some water temperature data is not on the website and some do not collect water temperature anymore.

Water temperature is a Water Survey of Canada output, not a product. The data are provided as-is. We have no standard operating procedures for data collection or processing. This means that sensor selection, calibration, and placement are not standardized, nor is there any systematic quality assurance or quality checking of the data.

Excerpt from document qREC-NA063-02-2020 WSC Official Products and Outputs (available in full on https://wateroffice.ec.gc.ca/ )

"WSC Products and Outputs are defined relative to the level of quality control to which they are subjected. A WSC Product is data or information that is routinely published by the Water Survey of Canada under formal bilateral Hydrometric Agreements with partners of the National Hydrometric Program, and that has been subject to the controls of all relevant quality management processes. A WSC Output is data or information that is regularly collected or computed by WSC, but that has not been subject to the controls of all relevant quality management processes."

image

image

image