New New Mexico (NM) gage stations data

[amabdallah]

This shapefile has NM gage stations. The attribute table points to a URL to query the time series data for each site. https://ose.maps.arcgis.com/home/webmap/viewer.html?useExisting=1&layers=ff2ee4069ba64819a980da46e767bb46

This is the current way of access data as an example (dont use it as it will be updated) http://meas.ose.state.nm.us/site.jsp?id=63&status=Y&type=S

At a minimum, we need to store the NM gage stations in the site's table. Monthly time series would be great, for now.

Kyle Onda at the IOW, is working with NM on new API access using the SensorThings way. Adel will follow up with Kyle in September or October
https://ose.newmexicowaterdata.org/FROST-Server/v1.1

[rwjam]

This shapefile has NM gage stations. The attribute table points to a URL to query the time series data for each site. https://ose.maps.arcgis.com/home/webmap/viewer.html?useExisting=1&layers=ff2ee4069ba64819a980da46e767bb46

At this time (09/25/2020) we can load at least the sites info, but probably not the time use data.

The time use data is not in an easily accessible format. We would have to manually download each site (min of 400 files).

[amabdallah]

The data seem to be available through the SensorThings API here https://catalog.newmexicowaterdata.org/dataset/ose-isc-real-time-water-measurement-information-system/resource/1e02b3fe-fb21-4a7d-aa7a-47e022be25cf

[rwjam]

Some notes:

• This 'should' be the first 1000 rows returned of the New Mexico SensorThings API specifically looking at 'Locatoins', then attaching on the 'Things-Datastreams' for those sites. ose api: https://st2.newmexicowaterdata.org/FROST-Server/v1.1/Locations?$filter=properties/agency%20eq%20%27OSE%27&$expand=Things/Datastreams
• It uses some HTML encoding [weblink], and SensorThings I think are hosted on servers that follow a FROST guide line [weblink]
  • eq = equals. For query string parameters in the URL. I think this a MS thing [weblink]
  • %27 = ', a single quote
  • %20 = ' ', a space
  • So the above translates into something like "https://st2.newmexicowaterdata.org/FROST-Server/v1.1/Locations?$filter=properties/agency = 'OSE'&$expand=Things/Datastreams"
• SesnorThings follows a data model group [weblink]. In this NM OSE case the SensorThings is looking at the 'Locations', then attaching 'Things->DataStreams 'related to those 'Locations'. You can swap out 'Location' and look at the other data model elements.

[amabdallah]

@rwjam Good news. Jake Ross from NM responded with this suggested access. Request readings for a single station within a date range: https://meterexttest.oseisc.org/extmrintake/api/meas_readings/30/1990-01-01/2023-01-01

*  Parameters
   *  Station_id: Integer ID matching the ID of the station on the MEAS website
   *  From_Date: YYYY-mm-dd format inclusive date of earliest returned results
   *  To_Date: YYYY-mm-dd format inclusive date of most recent returned results
       Data
*  Data should match the results provided on the publicly accessiblehttp://meas.ose.state.nm.us/ website. The date is provided in a JSON format.
*  Fields Provided:
   *  Station_ID: 148
   *  basin: Lower Pecos
   *  display_name: PDL-West
   *  district: 2
   *  Reading_ID: 279557207
   *  timestamp: 2021-06-15 09:00:00.000
   *  stage: 0.98000002
   *  discharge: 7.75

Sample JSON Data: `{"Station_ID":"148","basin":"Lower Pecos","display_name":"PDL-West","district":"2","Reading_ID":"279557207","timestamp":"2021-06-15 09:00:00.000","stage":"0.98000002","discharge":"7.75"}

Corresponding MEAS website data: http://meas.ose.state.nm.us/site.jsp?id=148&status=Y&type=S&dist=2&basin=Fort%20Sumner

Let's see how that goes if we loop over the Station_ID in the shapefile but only for sites with Jurisdiction = 'OSE'.

[rwjam]

Let's try this

• get shp file info from here link, particularly we want the site info & the Station_ID.
• then use this as an example. We nee Station_ID, a start & end date. Let's try 2000-01-01 - to 2023-01-01, for now
  • Ex script: https://meterexttest.oseisc.org/extmrintake/api/meas_readings/30/2000-01-01/2023-01-01
  • this was loading VERY slowly when I was playing with it. Possible too large of search date. Could be NM's servers can't handle this query.
  • this came back as a single string, not a JSON. I think I can still work with it, it's just one more step to get it out of the string format first.

[rwjam]

Jurisdiction

Looks like this API returns 15 minute intervals, which is too low for WaDE goals. We only want to work with Daily, at the minimum. Can't find any API documentation on our own, closest thing we found was this NM api store page.

[rwjam]

Let's use this API instead. Should return the information we need. Looks like we will have control over the site id, the start and end dates, the discharge data value (davg for our purposes), and the output type (we want csv). This loads straight into a DataFrame.

Ex: http://meas.ose.state.nm.us/ReportProxy?id=148&type=S&sDate=01/01/2011&eDate=01/24/2024&dischargeData=davg&rptFormat=HTML&sort=asc

[rwjam]

Pushed to Github: https://github.com/WSWCWaterDataExchange/MappingStatesDataToWaDE2.0/commit/8de1504d8ac76081a9081cc2f11344a9ae85a2b0