search

SlideRuleEarth / sliderule

Server and client framework for on-demand science data processing in the cloud
https://slideruleearth.io
Other
30 stars 12 forks source link

raster.sample() index is lastUpdated datetime rather than acquired datetime #444

Open scottyhq opened 2 weeks ago

scottyhq commented 2 weeks ago

I think this is a shortcoming of the national map API rather than sliderule, but wanted to document that it's not obvious timestamps returned from raster.sample are when the product was 'last updated' rather than when the data was acquired.

region = [  {"lon": -108.34, "lat": 38.89},
            {"lon": -107.76, "lat": 38.90},
            {"lon": -107.78, "lat": 39.26},
            {"lon": -108.36, "lat": 39.25},
            {"lon": -108.34, "lat": 38.89}  ]

geojson = earthdata.tnm(short_name='Digital Elevation Model (DEM) 1 meter', polygon=region)
gdf = raster.sample("usgs3dep-1meter-dem", [[-108.1,39.1]], {"catalog": geojson})
gdf
image

This is inconsistent with other APIs where 'time' is observation time of the data.

The timestamps are coming from here https://github.com/SlideRuleEarth/sliderule/blob/20a5b41617fe64def65a7ade7276bea109ee58c7/clients/python/sliderule/earthdata.py#L788

Unfortunately I don't think the national map api keeps track of the nominal acquisition date of a given 1m tile?...

elidwa commented 1 week ago

I am attaching an example of one item returned from a 1 meter 3dep TNM query. Please note that there are three dates provided: publicationDate, lastUpdateed, dateCreated. There are no acquisition dates available.

{ "title": "USGS one meter x23y432 CO MesaCo QL2 UTM12 2016", "moreInfo": "This is a tile of the standard one-meter resolution digital elevation model (DEM) produced through the 3D Elevation Program (3DEP). The elevations in this DEM represent the topographic bare-earth surface. USGS standard one-meter DEMs are produced exclusively from high resolution light detection and ranging (lidar) source data of one-meter or higher resolution. One-meter DEM surfaces are seamless within collection projects, but, not necessarily seamless across projects. The spatial reference used for tiles of the one-meter DEM within the conterminous United States (CONUS) is Universal Transverse Mercator (UTM) in units of meters, and in conformance with the North American Datum of 1983 (NAD83). All bare earth elevation values are in [...]", "sourceId": "5eaa4a0582cefae35a21ee8c", "sourceName": "ScienceBase", "sourceOriginId": null, "sourceOriginName": "gda", "metaUrl": "https://www.sciencebase.gov/catalog/item/5eaa4a0582cefae35a21ee8c", "vendorMetaUrl": "http://nationalmap.gov/3DEP/3dep_prodmetadata.html", "publicationDate": "2020-03-30", "lastUpdated": "2020-11-17T11:08:25.219-07:00", "dateCreated": "2020-04-29T21:46:13.083-06:00", "sizeInBytes": null, "extent": "10000 x 10000 meter", "format": "GeoTIFF", "downloadURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1m/Projects/CO_MesaCo_QL2_UTM12_2016/TIFF/USGS_one_meter_x23y432_CO_MesaCo_QL2_UTM12_2016.tif", "downloadURLRaster": null, "previewGraphicURL": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1m/Projects/CO_MesaCo_QL2_UTM12_2016/browse/USGS_one_meter_x23y432_CO_MesaCo_QL2_UTM12_2016_thumb.jpg", "downloadLazURL": null, "urls": { "TIFF": "https://prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1m/Projects/CO_MesaCo_QL2_UTM12_2016/TIFF/USGS_one_meter_x23y432_CO_MesaCo_QL2_UTM12_2016.tif" }, "datasets": [], "boundingBox": { "minX": -108.117274181, "maxX": -107.998021739, "minY": 38.9003581843, "maxY": 38.987435717 }, "bestFitIndex": 0.0, "body": "This is a tile of the standard one-meter resolution digital elevation model (DEM) produced through the 3D Elevation Program (3DEP). The elevations in this DEM represent the topographic bare-earth surface. USGS standard one-meter DEMs are produced exclusively from high resolution light detection and ranging (lidar) source data of one-meter or higher resolution. One-meter DEM surfaces are seamless within collection projects, but, not necessarily seamless across projects. The spatial reference used for tiles of the one-meter DEM within the conterminous United States (CONUS) is Universal Transverse Mercator (UTM) in units of meters, and in conformance with the North American Datum of 1983 (NAD83). All bare earth elevation values are in meters and are referenced to the North American Vertical Datum of 1988 (NAVD88). Each tile is distributed in the UTM Zone in which it lies. If a tile crosses two UTM zones, it is delivered in both zones. The one-meter DEM is the highest resolution standard DEM offered in the 3DEP product suite. Other 3DEP products are nationally seamless DEMs in resolutions of 1/3, 1, and 2 arc seconds. These seamless DEMs were referred to as the National Elevation Dataset (NED) from about 2000 through 2015 at which time they became the seamless DEM layers under the 3DEP program and lost the NED branding. Other 3DEP products include five-meter DEMs in Alaska as well as various source datasets including the lidar point cloud and interferometric synthetic aperture radar (Ifsar) digital surface models and intensity images. All 3DEP products are public domain.", "processingUrl": "processingUrl", "modificationInfo": "2020-11-17" },