Utah water use data - Githubissues

WSWCWaterDataExchange / MappingStatesDataToWaDE2.0

Manage all code to map and import state's data into WaDE 2.0

BSD 3-Clause "New" or "Revised" License

4 stars 1 forks source link

Utah water use data #46

Closed amabdallah closed 2 months ago

amabdallah commented 4 years ago

There is a download button for all data using CSV. Output unit should be either acre-feet or gallons (something to discuss)

Years range 1957-2019 Data https://waterrights.utah.gov/asp_apps/waterUseData/setFilters.asp

Metadata (click at display variables) https://waterrights.utah.gov/asp_apps/waterUseData/variableList.asp

amabdallah commented 4 years ago

This issue seems similar to this older one. Not sure if the data is the same and which one is easier to work with https://github.com/WSWCWaterDataExchange/MappingStatesDataToWaDE2.0/issues/42

rwjam commented 4 years ago

There is a download button for all data using CSV. Output unit should be either acre-feet or gallons (something to discuss)

Years range 1957-2019 Data https://waterrights.utah.gov/asp_apps/waterUseData/setFilters.asp

The CSV approach returns both System and Source Data within the same CSV file (the Source portion is burred near the bottom).

The JSON approach appears to have an error in it and does not return the entire data set. This might be something we should bring to their attention.

rwjam commented 4 years ago

Was able to map out and upload to our WaDE 2.0 system. Few issues that need to be addressed, should do together in person when we have the time.

amabdallah commented 3 years ago

Let's revisit this one as it includes diversions to agriculture

https://www.waterrights.utah.gov/cgi-bin/pubdump.exe?DBNAME=WRDB&SECURITYKEY=wrt2012access

rwjam commented 3 years ago

The data return is a link to a txt file. The link doesn't look act like an API, it's just a return on service kind of thing. Should be okay, if we use the CSV return, then place into excel with a text-to-columns.

The data itself is very dirty (see image). Will have to clean it up.

The timestamp value appear to either be in an unreadable format / a time stamp ID (rather than a value). I can't get a working format to work on it...

amabdallah commented 3 years ago

Right, it is probably like an FTP service. Sure, we'd use the CSV file similar to exporting water rights data for UT

rwjam commented 3 years ago

data imported.

rwjam commented 2 years ago

Reopening this issue.

We would like to go back and update this to also include agricultural & mining data timeseries data. For now keep it under the UT site-specific Public Supply Water Use data.

Get data from Water Use Data Portal (link). It should contain all the timeseries for all beneficial uses. The download merges System and Source data into the same tab, so you will have to separate the two manually.
- if the data does not contain unique beneficial uses (particularly all the M&I and the new agricultural & mining data, take a look at this API Variable Service (link) here.
Source data has lat & long site info to work with.
For System data, try to relate to the Utah Culinary Water Suppliers (link) shapefile data.

rwjam commented 1 year ago

Uploaded. Some odd links stretch across the state (caused by uncertain Source data that share location info). Jim followed up with a response saying you can either remove or ignore those odd source sites. Saying he will try and get a fix on that on future data releases.

"...those sources are located where the point is exactly at the public land survey system (PLSS) base location, which is the SE corner of temple square in SLC. These are likely points for which we don't have an accurate PLSS location and at some point were erroneously assigned a location at the base location. These lat/lon values should be ignored. I will talk to our technician that handles the water use reporting to see what it would take to get those updated to the correct locations"

amabdallah commented 1 year ago

Later next week, could you look if this data has a native URL that points to the state's data? If so, let's store it and also show it in the RShiny app. I've seen Utah landing pages for wells with historic records but not sure if they're available

rwjam commented 1 year ago

Later next week, could you look if this data has a native URL that points to the state's data? If so, let's store it and also show it in the RShiny app. I've seen Utah landing pages for wells with historic records but not sure if they're available

One issue is that we don't have a way to store native URLs for sites / records in our ssps tables in WaDE at this time. I can cheat for now with the RShiny app examples and just join that to the app data, just so we can see how that would look.