search

SAEON / odp-server

Source code for the SAEON Open Data Platform server components.
GNU Affero General Public License v3.0
0 stars 2 forks source link

Publishing somisana-related metadata to the ODP #3

Closed zachsa closed 11 months ago

zachsa commented 1 year ago

Hi @marksparkza ,

For SOMISANA, I would like to start publishing/archiving the daily, operational model output. I think this may be similar to the observations database publishing requirements. Basically every day we are producing N NetCDF files (one per model, currently there are 2 models). Am I correct that there are two options in integrating with the ODP in this regard?

  1. Publish a new metadata record for every netcdf file (but this will result in lots of records)
  2. Have a single catalogue entry per operational model that links to a landing page listing all the files

I need to implement this relatively soon - what are the steps to set this up on the ODP side of things? I think that option (2) is probably better as otherwise the ODP will fill up with similar-looking records. I remember that the ODP as it is currently deployed supports creating and updating records via an API. Do you mind if I use this in production?

marksparkza commented 1 year ago

Hi Zach,

Thanks for the inquiry. I think this will be a valuable integration going forward.

We will need to have a broader discussion to address matters such as data management, metadata quality control and integration testing, amongst other things.

I will chat to Leo about setting up a meeting. In the meantime, please take a look at the ODP wiki for further information about ODP policies and procedures.

marksparkza commented 11 months ago

This requirement is being addressed by a SAEON data integration working group focused on ocean modelling data.