search

iodepo / odis-arch

Development of the Ocean Data and Information System (ODIS) architecture
https://book.oceaninfohub.org/
26 stars 16 forks source link

ODIS interchange for the GOOS BioEco Portal / OCG / Ocean OPS / OBIS #402

Open pbuttigieg opened 3 months ago

pbuttigieg commented 3 months ago

This issue will further and consolidate the technical work to build an ODIS (meta)data flow to support interoperability between the GOOS OCG (xref #355), Ocean OPS, OBIS, the BioEco Portal, and the burgeoning EOV data system (xref #126 #385 etc)

The following will be the core implementation team: @pbuttigieg @pieterprovoost @kevin-obrien Magali Krieger (metadata expert) mkrieger@ocean-ops.org Thomas Latter tlatter@ocean-ops.org (front end and API developer)

The diagrams below illustrate the intended (meta)data flows and needs for alignment. The first task is to develop the ODIS-Arch specifications to facilitate the interoperation of these components, around its EOV exchange specification: https://book.oceaninfohub.org/thematics/variables/index.html

After testing, and verification, we will integrate the guidelines into GOOS EOV specification sheets.

pbuttigieg commented 3 months ago

image

pbuttigieg commented 3 months ago

A little rewiring in a follow-up with @pieterprovoost

The main change is to better show that OBIS and its BioEco Portal will/already do serve similar functions to OceanOps services, but focused on biodiversity (meta)data flows.

Through ODIS interoperation, all these modules can embed and leverage one another.

image

wardappeltans commented 3 months ago

The 10-odd GOOS networks are connected to OceanOPS, but in the BioEco world we do not have GOOS networks, but a zillion BioEco obs programmes and projects. So perhaps there should be a parallel flow from those programmes via the OBIS nodes network including central OBIS office. The OBIS data system can be placed at the same level as the ERDDAP.

pbuttigieg commented 3 months ago

The 10-odd GOOS networks are connected to OceanOPS, but in the BioEco world we do not have GOOS networks, but a zillion BioEco obs programmes and projects. So perhaps there should be a parallel flow from those programmes via the OBIS nodes network including central OBIS office. The OBIS data system can be placed at the same level as the ERDDAP.

From the ODIS point of view, that would make sense and also simplify the main data flow - if OBIS could be the hub, it would simplify things greatly. Of course, any system could connect to ODIS, share the EOV patterns, and be detectable by the BioEco Portal too, if they wanted / needed to.

jmckenna commented 3 months ago

I'm curious to hear from the technical teams how they each wish to pull from ODIS, I guess that can be discussed at the next technical meeting (direct SPARQL queries to the graph have proven to be so heavy, for example). PS> I think we should start using these workflow/Vue diagrams more often, very insightful.

pbuttigieg commented 3 months ago

I'm curious to hear from the technical teams how they each wish to pull from ODIS, I guess that can be discussed at the next technical meeting (direct SPARQL queries to the graph have proven to be so heavy, for example). PS> I think we should start using these workflow/Vue diagrams more often, very insightful.

Yes, my assumption is that they'll go for parquet or right at the JSON-LD / objects. We must then document more clearly how this can be done @fils