connect NCEI catalogue as an ODIS node

iodepo / odis-arch

Development of the Ocean Data and Information System (ODIS) architecture

https://book.odis.org/

29 stars 17 forks source link

connect NCEI catalogue as an ODIS node #361

Open jmckenna opened 11 months ago

jmckenna commented 11 months ago

sample record: https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.nodc:276264
data home: https://www.ncei.noaa.gov/products/microplastics
geo interface: https://experience.arcgis.com/experience/b296879cc1984fda833a8acc93e31476

jmckenna commented 10 months ago

Updates:

corrected spatial coordinates order in the template (see polygon coordinates at https://github.com/iodepo/odis-arch/pull/365/files )
some related important notes for schema.org's spatial support :
- schema.org expects lat,long coordinate pairs (y,x)
- this is opposite of most geo software (Shapely, etc.) which expects long,lat (x,y)
- GeoShape box is recommended, to avoid polygon issues, with a syntax of "box": "miny minx maxy maxx" (lower left, upper right)
```
    "@type": "GeoShape",
    "box": "46.37 -147.525 54.5617 -125.4467"
```

(we are currently updating all of our examples in the ODIS documentation, to match these rules)

pbuttigieg commented 8 months ago

The metadata looks good, with some caveats discussed in #365

However, the main issue blocking connection to ODIS is the lack of a sitemap.xml that points to the landing pages with embedded JSON-LD/schema.org.

There is a high-level sitemap here: https://www.ncei.noaa.gov/sitemap.xml

The easiest thing to do would be for NCEI to maintain a separate sitemap (e.g. https://www.ncei.noaa.gov/datasets/sitemap.xml) that lists all the landing pages of datasets (including their JSON-LD/schema.org metadata).

For the microplastics mission, we can try a more local solution where the microplastic team creates and exposes their own sitemap.xml that lists URLs to all landing pages + JSON-LD/schema.org that is relevant to them.

jmckenna commented 8 months ago

Updates regarding sitemap:

sitemap (template) generated, for NCEI: https://github.com/iodepo/odis-arch/blob/master/collection/tempHosting/data-ncei/sitemap.xml
script used to generate the sitemap: https://github.com/iodepo/odis-arch/blob/master/collection/scripts/ncei-sitemap.py

jmckenna commented 8 months ago

Yee/Eben fixed the sitemap, to include all 65 entries: https://github.com/iodepo/odis-arch/blob/master/collection/tempHosting/data-ncei/sitemap.xml
Also need to create an entry in the ODIS Catalogue (ODISCat) https://catalogue.odis.org

pbuttigieg commented 7 months ago

Metadata status examples :

https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.nodc:289355

https://www.ncei.noaa.gov/access/metadata/landing-page/bin/iso?id=gov.noaa.nodc:276264

Validation looks good, but some stanzas may need some refactoring