FRRF - no metadata header

[lbesnard]

Layer Fast Repetition Rate Fluorometry (FRRF) data collected on the RV L'Astrolabe

There is no metadata header. The list of variables is a bit confusing

FoL,FmL,FvL,Fv_Fm_ratio,SigL,pL,TauL,GAIN

[atkinsn]

As just discussed with @lbesnard we are aware ( @smancini ) that the IMAS collections are all like this, and we hope to help with improving this with them in the near future.

[xhoenner]

Where are we at with this @atkinsn ?

[atkinsn]

I think this one sits with @smancini , assigning someone to clean it up?

[leonardolaiolo]

@lbesnard or @atkinsn do you now what facility and subfacility are related to this Fast Repetition Rate Fluorometry (FRRF) data collected on the RV L'Astrolabe? Is it IMAS?

[atkinsn]

@leonardolaiolo this is not IMOS data, it's IMAS data. Also being discussed at present in this other issue - https://github.com/aodn/content/issues/420

[leonardolaiolo]

Thanks @atkinsn!

[leonardolaiolo]

So..I investigate this and I can create the headers for the CSV, I checked that the view created is good on my DB stack. On the other hand, I cannot test the changes are effective in my AODN portal stack, because, when I harvest from Geonetwork I need to update harvester External - AODN Portal IMAS (not Catalogue-imos portal records). This way I cannot see the collection in my AODN portal stack - nor download CSV. I'm then not sure how to proceed.

As reference, in data-services/PARAMETERS_MAPPING these changes are required in the following files: parameters_mapping.csv

IMAS;;FRRF;latitude;4;440;Degrees North;
IMAS;;FRRF;longitude;5;440;Degrees East;
IMAS;;FRRF;TEMP;959;429;;1
IMAS;;FRRF;PSAL;960;481;;1
IMAS;;FRRF;PAR;961;909;;1
IMAS;;FRRF;FoL;962;448;;1
IMAS;;FRRF;FmL;963;448;;1
IMAS;;FRRF;FvL;964;448;;1
IMAS;;FRRF;Fv_Fm_ratio;965;448;;1
IMAS;;FRRF;SigL;966;448;;1
IMAS;;FRRF;pL;967;448;;1
IMAS;;FRRF;TauL;968;910;;1
IMAS;;FRRF;GAIN;969;448;;1

parameters.csv

960;sea_surface_salinity;;sea_surface_salinity;;;
961;;;photosynthetically available radiation (wavelengths 400-700 nm);;;
962;;;minimum fluorescence in the light chamber of the FRRF (also used as a proxy for biomass);;;
963;;;maximum fluorescence in the light chamber of the FRRF;;;
964;;;variable fluorescence;;;
965;;;maximum quantum yield of photosystem II;;;
966;;;functional absorption cross section of PSII;;;this variable is a measure of the proportion of light harvested for photochemistry963;;;maximum fluorescence in the light chamber of the FRRF;;;
967;;;connectivity between PSII reaction centres under actinic light;;;
968;;;time constant of electron transport from PSII to PSI;;;
969;;;instrument\'s photomultiplier electronic sensitivity (1-256);;;

unit_view.csv

909;Mol photons m-2 s-1;;;2021-08-24
910;s-1;;;2021-08-24910

Furthermore, I wrote this to add in the PARAMETERS_MAPPING_harvester liquidbase second components Create parameters_mapping views:

    DROP VIEW IF EXISTS aodn_imas_fluorometry_metadata_summary;

    CREATE VIEW aodn_imas_fluorometry_metadata_summary AS
    WITH p AS (
    SELECT '# ' || variable_name || ',' || cf_standard_name || ',' || long_name || ',' || imos_vocabulary_name || ',' || uv.name || ',' || uv.short_name || ',' || uv.definition|| ',' || p.comment
        FROM parameters_mapping.parameters_mapping pm
        LEFT JOIN parameters_mapping.units uv ON pm.unit_id = uv.id
        LEFT JOIN parameters_mapping.parameters p ON p.unique_id = pm.parameter_id
          WHERE facility = 'IMAS' AND product = 'FRRF'
          ORDER BY variable_name || ',' || cf_standard_name || ',' || imos_vocabulary_name || ',' || uv.name || ',' || uv.short_name || ',' || uv.definition)
        SELECT '# data_column_name' || ',' || 'cf_standard_name' || ',' || 'long_name' || ',' || 'imos_vocabulary_name' || ',' || 'unit_name' || ',' || 'unit_short_name' || ',' || 'uv.definition' || ',' || 'comment'
    UNION ALL
        SELECT * FROM p WHERE "?column?" IS NOT NULL
    UNION ALL
        SELECT '#';