Connect ISA catalogue as an ODIS node

jmckenna commented 11 months ago

Initial summary:

ISA team to:
- review ODIS documentation
- create an entry in ODISCat
- provide a draft of JSON-LD for one dataset
OIH team to review draft and provide guidance

kmishiro-ISA commented 11 months ago

ISA reviews the ODIS documentation and creates the entry in ODISCat. But, we have found the "Deep Seabed and Ocean Database - International Seabed Authority (ODIS id: 893)," which was likely used for the OBIS database in the past. And we compile the oceanographic dataset while we draft JSON-LD by database colleagues. It may take time for the data compilation.

jmckenna commented 11 months ago

@kmishiro-ISA thank-you for the ODISCat entry. Later in the process we can use that same entry, and edit the ODIS-Arch URL, so that ODIS can find your records.

Next meeting goal: to develop a JSON-LD template together for one of your datasets (or review your existing JSON-LD).

kmishiro-ISA commented 10 months ago

This is to share that we created the very first draft of JSON-LD along the 'Dataset' JSON template on the OIH website using a few CTD sample data, which were directly input in the JSON file, but we can finalize our template in the next meeting with your advice. As the next step, we plan to incorporate the rest of the CTD and ADCP current data information from our database. deepdata_ctd.json

jmckenna commented 10 months ago

Thank-you @kmishiro-ISA, we can review this together in the next meeting.

jmckenna commented 10 months ago

Some important notes on spatialCoverage:

schema.org expects lat,long coordinate pairs (y,x)
this is opposite of most geo software (Shapely, etc.) which expects long,lat (x,y)

here is a working example, using polygon:


"spatialCoverage": {
    "@type": "Place",
    "geo": {
        "@type": "GeoShape",
        "polygon": "19.5495,157.5519 21.8613,157.5519 21.8613,158.37505 19.5495,158.37505 19.5495,157.5519"
    },
    "additionalProperty": {
        "@type": "PropertyValue",
        "propertyID": "http://dbpedia.org/resource/Spatial_reference_system",
        "value": "http://www.w3.org/2003/01/geo/wgs84_pos#lat_long"
    }
},

GeoShape box is recommended (instead of polygon shown above), with a syntax of "box": "miny minx maxy maxx" (lower-left, upper-right)
```
        "@type": "GeoShape",
        "box": "46.37 -147.525 54.5617 -125.4467"
```

jmckenna commented 10 months ago

template modified through today's meeting: deepdata_ctd_2023-11-09.json

kmishiro-ISA commented 10 months ago

This is the additional dataset (current meter data) using sample data for your review on format. deepdata_adcp_2023-12-06_km.json Our colleagues are compiling all the datasets based on the modified template for CTD.

jmckenna commented 9 months ago

code from today's review session: https://rustpad.io/#vnL1eO

jmckenna commented 9 months ago

Here is the formatted code from today's session:

deepdata_ctd_2023-12-07.json

{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@type": "Dataset",
    "@id": "https://data.isa.org.jm/tobe/aurl/thatpoints/tothis.json",
    "name": "Current meter data information acquired by ISA Contractor, BGR",
    "description": "Current meter sampling data captured in Oceanic Exploration Research BGR has implemented the following work as polymetallic sulphides exploration activities: [...] ",
    "url": "https://data.isa.org.jm/tobe/aurl/thatpoints/tothisinfo.html",
    "license": "This work is licensed under a Creative Commons Attribution (CC-BY) 4.0 License",
    "creditText": "Federal Institute for Geosciences and Natural Resources, Germany (BGR). Available : DeepData, International Seabed Authority https://data.isa.org.jm/ Accessed: [YYYY-MM-DD].",
    "dateCreated": "2023-12-06T18:31:00.000Z",
    "temporalCoverage": "2018-10-19T00:01:00.000Z/2019-11-05T07:01:00.000Z",
    "keywords": [
        "ADCP",
        "Currentmeter",
        "other keywords",
        "here"
    ],
    "measurementTechnique": "this can be text or a URL pointing to a document",
    "variableMeasured": [
        {
            "@type": "ProperyValue",
            "name": "Current Speed",
            "value": "0.265"
        },
        {
            "@type": "PropertyValue",
            "name": "Current Direction",
            "description": "Current Direction: current direction is defined towards. 0 degrees points towards North/positive Y, 90 degrees towards East/positive X, 180 degrees towards South/negative Y, and 270 towards West/negative X",
            "value": "214.876"
        },
        {
            "@type": "PropertyValue",
            "name": "Currentmeter Depth",
            "description": "Depth in water column for currentmeter device reading",
            "value": "2614"
        },
        {
            "@type": "PropertyValue",
            "name": "Coordinates",
            "description": "Coordinate of currentmeter data",
            "value": "68.5890,-21.2480"
        }
    ],
    "distribution": {
        "@type": "DataDownload",
        "contentUrl": "",
        "encodingFormat": "application/zip"
    },
    "spatialCoverage": {
        "@type": "Place",
        "geo": {
            "@type": "GeoShape",
            "polygon": "-23.0000 67.0000, -23.0000 71.0000, -20.0000 71.0000, -20.0000 67.0000"
        },
        "additionalProperty": {
            "@type": "PropertyValue",
            "propertyID": "http://dbpedia.org/resource/Spatial_reference_system",
            "value": "http://www.w3.org/2003/01/geo/wgs84_pos#lat_long"
        }
    },
    "provider": [
        {
            "@type": "Organization",
            "legalName": "International Seabed Authority",
            "name": "International Seabed Authority",
            "url": "https://oceanexpert.org/institution/13169"
        },
        {
            "@type": "Organization",
            "legalName": "Federal Institute for Geosciences and Natural Resources, Germany (BGR)",
            "name": "Federal Institute for Geosciences and Natural Resources, Germany (BGR)",
            "url": "https://oceanexpert.org/institution/11193"
        }
    ]
}

kmishiro-ISA commented 3 months ago

Please kindy review the attached JASON-LD: DeepData+LD.json

kmishiro-ISA commented 3 months ago

This is the JSON-LD for a single dataset:

{
            "@type": "Dataset",
            "name": "Government of the Republic of Korea - PMS",
            "distribution": {
                "@type": "DataDownload",
                "contentUrl": "
[https://data.isa.org.jm/isa/map/download/koreapms12016_env_watercolumnzip"](https://data.isa.org.jm/isa/map/download/koreapms12016_env_watercolumnzip%22)
,
                "encodingFormat": "application.zip"
            },
            "url": "
https://data.isa.org.jm"
,
            "temporalCoverage": "2011",
            "geo": {
                "@type": "GeoCoordinate",
                "longitude": "-15.39500000,",
                "latitude": "15.39500000,6",
                "additionalProperty": {
                    {
                        "@type": "PropertyValue",
                        "name": "Water Depth",
                        "QuantitativeValue": {
                            "minValue": 1000.0,
                            "maxValue": 2787.0
                        }
                    },
                    {
                        "@type": "PropertyValue",
                        "name": "pH",
                        "QuantitativeValue": {
                            "minValue": "7.83118741058655",
                            "maxValue": "7.90128755364807"
                        }
                    }
                }
            }
        }

jmckenna commented 3 months ago

@kmishiro-ISA using the above template that we had created together earlier, here is your latest sample, updated for many missing/incorrect parameters:

{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@type": "Dataset",
    "@id": "https://data.isa.org.jm/tobe/aurl/thatpoints/tothis.json",
    "name": "Government of the Republic of Korea - PMS",
    "distribution": {
        "@type": "DataDownload",
        "contentUrl": "https: //data.isa.org.jm/isa/map/download/koreapms12016_env_watercolumnzip",
        "encodingFormat": "application/zip"
    },
    "url": "https: //data.isa.org.jm",
    "temporalCoverage": "2011",
    "spatialCoverage": {
        "@type": "Place",
        "geo": {
            "@type": "GeoCoordinates",
            "longitude": "-15.39500000",
            "latitude": "6"
        }
    },
    "variableMeasured": [
        {
            "@type": "PropertyValue",
            "name": "Water Depth",
            "description": "longer description",
            "minValue": 1000.0,
            "maxValue": 2787.0
        },
        {
            "@type": "PropertyValue",
            "name": "pH",
            "description": "longer description",
            "minValue": "7.83118741058655",
            "maxValue": "7.90128755364807"
        }
    ]
}

@pbuttigieg can you review here also? (this does now validate)

jmckenna commented 3 months ago

@pbuttigieg re-pinging you here (see above)

kmishiro-ISA commented 3 months ago

Thank you, Jeff, for your review, and yes, @pbuttigieg can also review this if any, that would be appreciated.

pbuttigieg commented 3 months ago

looks good @kmishiro-ISA

some small issues fixed below:

contentUrl had an extra space.
I'm assuming that the name and description field will be made more informative
in general, one should maintain decimal points in the lat lon to roughly express accuracy, but this is more a question of your data sources than your data sharing
the url value should direct users to a page about the dataset described in the metadata, not a general page. if this isn't possible, the link to the data portal itself is acceptable .
for future work, adding units and semantics for variables would enhance these records

@jmckenna otherwise I think we can move on to testing ingest

{
    "@context": {
        "@vocab": "https://schema.org/"
    },
    "@type": "Dataset",
    "@id": "https://data.isa.org.jm/tobe/aurl/thatpoints/tothis.json",
    "name": "Government of the Republic of Korea - PMS",
    "distribution": {
        "@type": "DataDownload",
        "contentUrl": "https://data.isa.org.jm/isa/map/download/koreapms12016_env_watercolumnzip",
        "encodingFormat": "application/zip"
    },
    "url": "https: //data.isa.org.jm",
    "temporalCoverage": "2011",
    "spatialCoverage": {
        "@type": "Place",
        "geo": {
            "@type": "GeoCoordinates",
            "longitude": "-15.39500000",
            "latitude": "6"
        }
    },
    "variableMeasured": [
        {
            "@type": "PropertyValue",
            "name": "Water Depth",
            "description": "longer description",
            "minValue": 1000.0,
            "maxValue": 2787.0
        },
        {
            "@type": "PropertyValue",
            "name": "pH",
            "description": "longer description",
            "minValue": "7.83118741058655",
            "maxValue": "7.90128755364807"
        }
    ]
}

anselcadien commented 3 months ago

@jmckenna we've added the embedded json+ld to the below web page.

https://www.isa.org.jm/deepdata-database-2/

It's not in the footer so not sure how that will impact the ingest.

kmishiro-ISA commented 4 hours ago

@jmckenna @pbuttigieg

This is to share the current JSON-LD, which was embedded in the ISA page for the test: JSON-LD-ISA-OIH.json

iodepo / odis-arch

Connect ISA catalogue as an ODIS node #356