OR Streamflow data - Githubissues

WSWCWaterDataExchange / MappingStatesDataToWaDE2.0

Manage all code to map and import state's data into WaDE 2.0

BSD 3-Clause "New" or "Revised" License

4 stars 1 forks source link

OR Streamflow data #119

Closed rwjam closed 1 year ago

rwjam commented 2 years ago

(08/25/2022) Looks like OR 'might' have some stream flow data. However, there is no way to mass download this data. Will need to keep an eye on this and see if that changes in the future. https://www.oregon.gov/OWRD/pages/index.aspx

amabdallah commented 2 years ago

I just emailed Jordan Beamer asking about dynamic access to this data + a shapefile of the stations https://apps.wrd.state.or.us/apps/sw/hydro_report/

I'd say this data is available, its a matter of time for us to get access to it.

rwjam commented 2 years ago

Mellony Hoskinson & Ryan Andrews of the Oregon Water Resources Department shared with us via email a OWRD_gages.zzz and an extract_mdfscript.R file with enough useable information. Saved info to local files for now.

manually rename OWRD_gages.zzz to OWRD_gages.zip so we can unzip the provided shapefile.
use provided OWRD_gages shapefiles for site information.
Use API to retrieve timeseries info. Use gages shapefile info as parameters. station_nbr = station_nbr. periodof = start_date. period_of_1 = end_date
EX: https://apps.wrd.state.or.us/apps/sw/hydro_near_real_time/hydro_download.aspx?station_nbr=10366000&start_date=3/1/1910&end_date=9/30/2011&dataset=MDF&format=html

amabdallah commented 2 years ago

Is the task complete or still in progress? Nice work. Thanks to the toggle button that hides no record sites, I see that we have five sites (out of dozens) with time series data. This is a deliverable for this quarter, and with the holidays coming up, I want to make sure we ask Oregon folks if we need anything as soon as we can.

rwjam commented 1 year ago

There should be MUCH more working sites than just 5. Will review data / code and see what is causing this error.

rwjam commented 1 year ago

Issue solved. Missing a few years for the ReportYearCV table. Prevented full dataset from uploading to WaDE. Fixed issue and reupload data.

amabdallah commented 1 year ago

Thanks to this summary, I remembered that during one of our calls with Oregon, they shared that stream data pre-1950s could be more reliable. So let's take down all time series data pre-1950. So the start is 01/01/1950

rwjam commented 1 year ago

Double checking the logic here.

Is that shared stream data pre-1950s that is more reliable the data we are using here in WaDE? Or is that a separate dataset?
Are we at a disadvantage of just leaving the pre 1950 data in WaDE alone? I'm not following why we should remove data before a certain point. We should not be mixing dataset together ourselves, that would be the state's responsibilities.

amabdallah commented 1 year ago

So its the same data source and data we have, we're just trimming (removing) data before Jan 1950. The OR state folks were not comfortable in sharing data pre 1950 because the quality is not guaranteed or QA/QC ed. They suspected the instruments had issues here and there before then.