WSWCWaterDataExchange / MappingStatesDataToWaDE2.0

Manage all code to map and import state's data into WaDE 2.0
BSD 3-Clause "New" or "Revised" License
4 stars 1 forks source link

NE water rights data #29

Open amabdallah opened 4 years ago

amabdallah commented 4 years ago

Not sure yet if their data is publically accessible but here is a place to start looking
https://dnr.nebraska.gov/data

tsga commented 4 years ago

They have the following three groups of data (of interest to us) among many other data, and I was able to download two of them through their web-GIS API:

  1. Registered Wells-DNR (I downloaded Registered_WellsDNR.csv) with fair amount of information on ground water PODs and owners

  2. Dams GIS data (I downloaded Dams.csv) on dams and their storage and their owners...

  3. The "Nebraska Surface Water Rights Data Retrieval" however requires filling a for a specific water right and there appears no clear method to bulk download these data.

amabdallah commented 4 years ago

Beneficial uses dictionary from WaDE 1.0

LU_BENEFICIAL_USE.txt

amabdallah commented 4 years ago

Metadata dict https://nednr.nebraska.gov/media/WaterRights/SurfaceWaterWebSimpleSearch.pdf

Tabulated data (no long and lat coordinates) https://nednr.nebraska.gov/Dynamic/WaterRights/WaterRights/SurfaceWaterRightsSearch?ShowPermitNumberFilter=true&ShowSectionTownshipRangeFilter=true&ShowContactFilter=true&ShowFacilityNameFilter=true I copied the tabular data into this text file. It can be loaded into Excel [Nebraska_WR.txt] (https://github.com/WSWCWaterDataExchange/MappingStatesDataToWaDE2.0/files/4560989/Nebraska_WR.txt)

SwaggerHub API https://nednr.nebraska.gov/IwipApi/swagger/ui/index#/

AllSurfaceWaterPoints This one is the important one to get water rights data from. Not sure how many pages are there but we need to look into Tsganeh's code that iterates over API pages and merges all returned data together

https://nednr.nebraska.gov/IwipApi/api/v1/WaterRights/AllSurfaceWaterPoints?page=1

The API returns JSON data and this screenshot views its elements. It's lacking the long and lat coordinates which are needed for WaDE 2.0 image

rwjam commented 4 years ago

I'm able to parse our these APIs into workable tables and data frames using a simple python code.

I'm having trouble understanding the difference between each API though, I feel the cover the same info (i.e., what id the difference between well, adminwells, and a water right?)

image

amabdallah commented 3 years ago

Note: water rights usually in Nebraska are referred to as surface water (not including groundwater, often)

amabdallah commented 3 years ago

So for groundwater permits, this API call seems to get what we want but we need to page through it. I saw on this app ~200 Ks of rows. The page seems to get 1000 records, so we have 200 pages out there to loop through and append into a single data frame. https://nednr.nebraska.gov/IwipApi/swagger/ui/index#!/Wells/Wells_AllWells

Note: I think this data is exempt of priority date but still comes with a flowrate image

amabdallah commented 3 years ago

This web application has the POD data and supports expoting to CSV from its "Search" and Query tool but it downloads through pages of 1000 rows at a time. There seem to be 20 pages which is too much to do manually.

https://prodmaps2.ne.gov/Html5DNR/index.html?viewer=surfacewater

rwjam commented 3 years ago

This web application has the POD data and supports expoting to CSV from its "Search" and Query tool but it downloads through pages of 1000 rows at a time. There seem to be 20 pages which is too much to do manually.

https://prodmaps2.ne.gov/Html5DNR/index.html?viewer=surfacewater

Based on the meeting with Nebrasks on 08/14/2020, it sounded like this data is out of date. Their advice was to use the API instead.

rwjam commented 3 years ago

They claim around 12,000 entries / rows for surface water, and around 257,000 entries / rows for ground water.

Their API seems to return duplicate rows. Their 'AllSurfaceWaterPoints' API only contains 8790 non-duplicate rows. Tested with different return sizes too.

rwjam commented 3 years ago

Their surface water data does not contain lat or long values. Waiting on static file from department (from the 08/14/2020 meeting, they said they could email a static file with this info to us).

Their well data does not contain priority date info.

amabdallah commented 3 years ago

I just sent a follow up to Jennifer and Kim about the surface water PODs shapefile.

For groundwater, we will use the new proposed field ~ "ExemptOfPriorityDateAmount" to allow storing their GW data with no priority dates. For testing purposes, you can add a dummy date like 01/01/1900 for all of them.

amabdallah commented 3 years ago

Let's plan on importing the GW permits after DPL finished their database update.

rwjam commented 3 years ago

Date uploaded to WaDE. Waiting for state agency review.

rwjam commented 3 years ago

POU site data available at here. Should be able to link to POD and water right record via the RightID field. Was able to retrieve shapefile.

image

rwjam commented 2 years ago

Something to keep an eye on, an API update service for NE DNR. image

amabdallah commented 1 year ago

I was trying to see why NE doesn't have owner name classification (all is unspecified). After looking into their API, they seem to provide the owner info under a separate JSON branch called: Contacts. Let's try to add those owner names, probably concatenate first and last name with a space between them Example call for page1 https://nednr.nebraska.gov/IwipApi/api/v1/WaterRights/AllSurfaceWaterPoints?page=1

image

amabdallah commented 1 year ago

Nebraska now has a landing page for each water right. https://nednr.nebraska.gov/dynamic/WaterRights/WaterRights/SurfaceWaterRightsSearchIndex

They also have what appears to be a POU later connected to each water right. https://www.nebraskamap.gov/datasets/surface-water-right-boundaries/explore?location=41.452406%2C-99.687989%2C7.71&showTable=true

Example landing page https://nednr.nebraska.gov/dynamic/WaterRights/WaterRights/SWRDetailPage?RightID=2

amabdallah commented 1 year ago

Does NE data come with Irrigated acreage? their landing page comes with this data https://westdaatstaging.westernstateswater.org/details/right/NVwr_WR2559

image

rwjam commented 1 year ago

https://westdaatstaging.westernstateswater.org/details/right/NVwr_WR2559 This is a NV site, not a NE site.

It think you meant to share this link: https://westdaatstaging.westernstateswater.org/details/right/NEwr_WR59 Native Landing Page: https://nednr.nebraska.gov/dynamic/WaterRights/WaterRights/SWRDetailPage?RightId=59

Possibly. There is no metadata available for the NEwr data, so we are guessing at the inputs.

In the source input data for this wr, there is a 70 value associate with a rate. But there is also CurrentTotalAcres that 'might' imply Irrigated acreage. This is a question for NE. image