Harvesting authoritative data catalogs outside OpenGeometadata.org

[thatbudakguy]

Mostly DCAT and STAC catalogs:

https://data.gov/

https://www.census.gov/data/developers/updates/new-discovery-tool.html

https://developers.google.com/earth-engine/datasets/ (Comes with sample code that could be used to "open in Google Earth Engine")

https://data.humdata.org/

https://samapriya.github.io/awesome-gee-community-datasets/

https://earthengine-stac.storage.googleapis.com/catalog/catalog.json

https://stacindex.org/

https://earth.esa.int/eogateway/catalog

[thatbudakguy]

blocked on analysis of how to convert this to geoblacklight JSON, where to store it, etc. might be a geocombine thing.

[thatbudakguy]

There are two buckets of work here:

1. DCAT format catalogs, which are mostly gov data of all kinds and varying quality
2. STAC catalogs, which are mostly temporally varying, cloud-optimized geoTIFF data (e.g. from satellites)

Both may be updated at a much higher frequency than other data we index.

We would love to offer this data (perhaps starting small) through opengeometadata; for now we can track the work here.

The STAC catalogs are likely blocked on #867.