search

creare-com / podpac

Pipeline for Observational Data Processing Analysis and Collaboration
https://podpac.org
Apache License 2.0
44 stars 6 forks source link

Wrap Data from NDMI Node #270

Closed mpu-creare closed 5 years ago

mpu-creare commented 5 years ago

Description We need to be able to pull data from here https://droughtmonitor.unl.edu/ to compare to our SMAP version.

This might be a good place to start: https://droughtmonitor.unl.edu/Data/DataTables.aspx

drc-creare commented 5 years ago

There are a lot of different ways to get at this data. Given what I think we want to do with it, I think the most promising is actually via WMS:

http://ndmc-001.unl.edu:8080/cgi-bin/mapserv.exe?map=/ms4w/apps/usdm/service/usdm_20190528_wms.map&SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&LAYERS=usdm20190528&WIDTH=640&HEIGHT=480&crs=EPSG:3857&styles=default&format=image/png&bbox=-18367715.9809,1689200.13961,-6679169.4476,15538711.0963

Using the colormap which can be found at the bottom of:

https://droughtmonitor.unl.edu/Data/Metadata.aspx

A number of geospatial data file types can be retrieved bassed on date, including WMS https://droughtmonitor.unl.edu/Data/GISData.aspx:

There is a REST API https://droughtmonitor.unl.edu/WebServiceInfo.aspx#comp that returns JSON. However, it looks like a single query is for a specific location and a range of dates. I think we want all locations for specific dates:

https://usdmdataservices.unl.edu/api/[area]/[statistics type]?aoi=[aoi]&startdate=[start date]&enddate=[end date]&statisticsType=[statistics type]

https://droughtmonitor.unl.edu/Data/Metadata.aspx

drc-creare commented 5 years ago

@mpu-creare Do we already have a WMS Node? If so, perhaps we can use the WMS feed for the NDMI data.

drc-creare commented 5 years ago

It looks like not all dates are valid, but we can use the REST API to retrieve all valid dates for a given year. I think they mention that the REST API is limited to one year at a time:

curl "https://usdmdataservices.unl.edu/api/USStatistics/GetDroughtSeverityStatisticsByArea?aoi=us&startdate=1/1/2019&enddate=1/1/2020&statisticsType=1"
mpu-creare commented 5 years ago

This is perfect. We can use the WMS endpoint for the slippy map (leaflet, no PODPAC required), and we can use the REST API endpoint for the line plot -- so we need a node for that.

drc-creare commented 5 years ago

The REST API is an interesting use case for PODPAC. Its clear how time would be represented as native coordinates, location is not as clear. The REST API can be queried by several different area types in order to retrieve statistics for those areas:

I believe these are all polygon regions rather than grids. I think county is probably the most fine grained, though I don't know what may of these are. One of the Army Corps area types might be more fine grained.

We might be able to get the polygon definitions out of one of the vector GIS files. Given polygons, its possible to determine which county a point would be in, or how a grid cell intersects with the county polygons. Though, I dont think this is in PODPAC right now. There may also be an API somewhere that will tell you what county a point is in. Either way, not sure what the native (spatial) coordinates would be. These could be supplied to the constructor of the NDMI Node and we could just use the SMAP grid for our application.

drc-creare commented 5 years ago

FCC has an API for retrieving the FIPS county code ofr a lat/lon point:

https://geo.fcc.gov/api/census/

NDMI uses the FIPS code. FCC API TOS states that:

"All services which utilize or access the API should display the following notice prominently within the application: “This product uses the FCC Data API but is not endorsed or certified by the FCC”...."

They dont say what coordinate system they are using. I'd guess WGS84 lat/lon in degrees.

drc-creare commented 5 years ago

Feature Request: Polygon to Grid interpolation.

mpu-creare commented 5 years ago

I'll take over. Seems like we could do this on the Javascript side pretty easily.

mpu-creare commented 5 years ago

We did this in JavaScript, so it's done.