search

EMODnet / esas2obis

Darwin Core mapping of ESAS data for publication to OBIS
MIT License
0 stars 0 forks source link

Completed project deliverables #36

Closed peterdesmet closed 1 year ago

peterdesmet commented 1 year ago

The EMODnet Biology service provider contract between INBO and VLIZ listed 3 deliverables. Here's how these were met:

Provide an overview of a complete mapping between the ESAS database held at ICES and DwC EventCore, documented in a GitHub repository

A complete mapping was made for the ESAS database hosted at ICES following OBIS best practices. It transforms all ESAS data/fields into an Event Core, Occurrence Extension and Extended Measurement or Facts (EMOF) extension. Much of the information is expressed in the EMOF, with extensive links to BODC controlled vocabularies for measurementTypeID, measurementValueID and measurementUnitID.

The transformation is expressed in 3 SQL files and can be run in R using the dwc_mapping.Rmd script. It starts from a download of public ESAS data, which can be initiated at https://esas.ices.dk/inventory.

The mapping, code to run it and documentation are managed in this self-contained repository: https://github.com/EMODnet/esas2obis

Complete a first data transfer of ESAS data from ICES to VLIZ (through INBO) in DwC EventCore format

A first version of the dataset was delivered on 2022-12-02. This was reviewed by the EurOBIS staff and resulted in an accepted version of the dataset that was delivered on 2023-02-21: https://ipt.vliz.be/upload/resource?r=esas&v=1.3. It contains all public data held in the ESAS database at that point, amounting to:

This dataset is in the process of being included in EMODnet Biology, OBIS and GBIF.

Document the data flow and all instructions for future updates of ESAS data from ICES to EurOBIS/EMODnet Biology

The steps to republish the dataset are documented in the README of the repository: https://github.com/EMODnet/esas2obis#workflow. The repository also contains a sample of the data that should remain the same. It can be used to verify that the transformation does not contain any errors.

The entire Darwin Core transformation (in addition to being documented as SQL files) is documented/summarized in the README as well: https://github.com/EMODnet/esas2obis#darwin-core-transformation

peterdesmet commented 1 year ago

I tested the entire process on a new ESAS download that contains slightly more data. It ran without issues. The resulting data has increased slightly:

I won't republish the dataset, as the current one is still being processed by EuroOBIS. The new data will be included in future (e.g. yearly) updates.