Preparatory work

[TAlonglong]

Just looking at how the wms, data_access is done in mmd and the resulting xml file https://github.com/metno/mmd/blob/master/mmd_utils/nc_to_mmd.py#L351-L370

So its a raw harvesting of the datasets in the netcdf for the layers with the thredds wms server/filename as the server.

How is this done in NBS? @ferrighi do you have any xml fil how this looks? I guess it must me the same?

An example:

  <mmd:data_access>
    <mmd:type>OGC WMS</mmd:type>
    <mmd:description>OGC Web Mapping Service, URI to GetCapabilities Document.</mmd:description>
    <mmd:resource>https://thredds.met.no/thredds/wms/remotesensingsatellite/polar-swath/2021/08/26/noaa18-avhrr-20210826103908-20210826105303.nc?service=WMS&amp;version=1.3.0&amp;request=GetCapabilities</mmd:resource>
    <mmd:wms_layers>
        <mmd:wms_layer>toa_bidirectional_reflectance</mmd:wms_layer>
        <mmd:wms_layer>toa_bidirectional_reflectance</mmd:wms_layer>
        <mmd:wms_layer>toa_bidirectional_reflectance</mmd:wms_layer>
        <mmd:wms_layer>toa_brightness_temperature</mmd:wms_layer>
        <mmd:wms_layer>toa_brightness_temperature</mmd:wms_layer>
        <mmd:wms_layer>toa_brightness_temperature</mmd:wms_layer>
    </mmd:wms_layers>
  </mmd:data_access>

[ferrighi]

You can find the NBS xml here: https://gitlab.met.no/FoU-HI-RS/nbs-md-records/ or directly on the site (https://satellittdata.no/en/metsis/search) with the button export mmd.

@ElodieFZ has the overview of the routine and scripts that creates mmd files for NBS. She can share that info with you.

[TAlonglong]

Here is some of what I talked about @mortenwh

Not sure if we need to find a way around it in the mmd files of the mmd files strictly should be harvested from the netcdf files

[mortenwh]

We should probably change nc_to_mmd.py.. Or make a "post-processing" tool to allow changing some fields?

[TAlonglong]

Maybe a postprocessing tool would be a way to go. Else I think it will complicate the already nc to mmd even more.

[TAlonglong]

Thank you @ferrighi. I looked at the page yesterday, but did not "see" the export mmd button. Very nice!

But I guess the data on NBS are already gridded into a projection, and that makes the life a bit easier.

[TAlonglong]

This work has led to a prototype of the wanted setup. A docker container has been made with mapserver and apache2. https://gitlab.met.no/trygveas/okd-satellite-mapserver-image. No caching is applied yet. So this needs to be implemented as needed at a later stage. This can be mapcache, mapproxy (https://mapproxy.org/) or general cache like varnish. Also a k3s(minicube) setup has been setup at my laptop inside a vagrant instance. This is very similar to the vagrant-senda which has been used as a stating point. Thank you Arnulf for this setup. https://gitlab.met.no/trygveas/vagrant-senda-mapserver With this setup you need to sync data inside the persistent storage, generate unique map file for this data and update the mmd xml file with the new OGC WMS service. Please see script https://gitlab.met.no/trygveas/vagrant-senda-mapserver/-/blob/master/scripts/py-mmd-edit-resource.py how this can be done.