OceanGlidersCommunity / OG-format-user-manual

OceanGliders format and vocabularies
15 stars 13 forks source link

Raw counts variables in OG1 #261

Open callumrollo opened 1 month ago

callumrollo commented 1 month ago

Moderator: @OceanGlidersCommunity/format-maintainers

We include the raw counts from several sensors, notably flourometers, as well as their scaled values. How should these terms be requested in OG1 vocab?

I see a couple of examples with the prefix FLOU that refer to the counts e.g. for CHLA we have Chlorophyll-A

https://vocab.nerc.ac.uk/collection/OG1/current/CHLA/

and counts

https://vocab.nerc.ac.uk/collection/OG1/current/FLUOCHLA/

Should this be the same for all raw sensor counts? What does the FLUO prefix mean?

callumrollo commented 1 month ago

As a follow question to this, FLUOCHLA in OG1 has mapping that says it is the same as FCNTRW01 in P01

https://vocab.nerc.ac.uk/collection/P01/current/FCNTRW01/

FCNTRW01has a more descriptive description "Raw signal (counts) of instrument output by in-situ chlorophyll fluorometer" rather than "Chlorophyll-A signal from fluorescence sensor" from OG1/FLUOCHLA. Can this description be used in the long_name metadata of an OG1 file? Or must it be the description from the OG1 concept?

danibodc commented 1 month ago

Hi @callumrollo, to request new OG1 terms, please create an issue ticket in the same area as this ticket (https://github.com/OceanGlidersCommunity/OG-format-user-manual/issues/) with the label 'Vocab'. The proposal can then be discussed and agreed amongst the community, and once agreed it can be assigned (either by me or @emmerbodc) to a member of the vocabulary management group at BODC. Most likely me :)

Alternatively we could look to grant the relevant roles certain members of OGDMTT, so that once a term has been agreed within the community, they would then be able to load new terms (and mappings to P01) themselves via our vocabulary editor tool: https://vocab.nerc.ac.uk/editor/. I think this would be the preferred route in the long term, and we can provide training on how to use this tool as needed.

If you have identified any 'raw' type terms missing from OG1 that should be created, whether they be counts, signals, voltages etc, feel free to list them below.

R.e. the identifiers 'CHLA' and 'FLUOCHLA', I'm not 100% sure of the initial intention behind the choice of identifier here, but seems it was to provide a distinction between the completely raw counts straight from the instrument, and calibrated concepts (manufacturer, laboratory or sample calibration).

Note that this is not the intended use of identifiers - the collection should not really have been created with identifiers that might infer meaning. Identifiers should ideally be as opaque as possible so as not to cause unnecessary ambiguity, as we are seeing here. I believe the OG1 collection was created following the format of the the Argo R03 collection. I notice R03 contains similar concepts http://vocab.nerc.ac.uk/collection/R03/current/FLUORESCENCE_CHLA/, and https://vocab.nerc.ac.uk/collection/R03/current/CHLA/, which may have been what the OG1 identifiers were based on.

We cannot alter identifiers once they are live, but we can alter the preferred labels and/or definitions if deemed appropriate. So to answer your second question, yes we could quite easily update the preferred label for FLUOCHLA from 'Chlorophyll-A signal from fluorescence sensor' to match the equivalent P01 label 'Raw signal (counts) of instrument output by in-situ chlorophyll fluorometer'.

However, to be more aligned with existing 'raw' terms in OG1, I would perhaps suggest a similar label to the equivalent R03 term: 'Raw signal from chlorophyll-a fluorometer'.

This would just need to be agreed amongst the community first, then the term can be modified. The existing term can also be modified via the vocabulary editor tool, if OGDMTT think it would be useful to assign the necessary roles to team members.

callumrollo commented 1 month ago

Thanks for the thorough explanation Dani! Still trying to get my head around how vocabs work. I'm trying to pose all the questions I struggle with as github Issues so we can refer to them later if/when others have questions about the vocabs

callumrollo commented 1 month ago

We distribute raw/counts data for the following OG1 concepts that do not appear to have a corresponding raw/counts concept:

danibodc commented 1 month ago

@callumrollo r.e. BBP700, are you referring to completely raw counts, or has the vendor supplied scale factor been used to calculate the volume scattering coefficient 'Beta' (per metre per steradian)?

If the latter, we currently have a number of existing 'Beta' terms where the calculation using the scale factor has been applied: https://vocab.nerc.ac.uk/search_nvs/OG1/?searchstr=beta&options=identifier,preflabel,altlabel,definition,status_accepted&rbaddfilter=inc&searchstr2=

callumrollo commented 1 month ago

In the case of BBP700 it is the raw counts (unitless) from the sensor before the scale factor has been applied

emmerbodc commented 1 week ago

Discussed in vocab meeting 06/09/24 Agreed to use similar R03 term: 'Raw signal from chlorophyll-a fluorometer'. @action @danibodc to update term

Action @emmerbodc to reach out to Callum and Andy to obtain more information on turbidity - suggested updating existing - https://vocab.nerc.ac.uk/collection/OG1/current/TURB/ to link to units output from sensor

Agreed that we need raw counts for backscatter @danibodc to propose new ones based on R03

danibodc commented 5 days ago

Update to FLUOCHLA has been made and will be live on the NVS from tomorrow morning.

For the new raw backscattering term at 700 nm, please see the below proposed term:

Proposed identifier: RPB700 Proposed preferred label: Raw signal from backscattering sensor Proposed alternative label: RAW_BACKSCATTERING700 Proposed definition: Raw backscattering sensor signal (count) at 700 nm wavelength. Volume scattering scale factor has not been applied.

emmerbodc commented 2 days ago

I'm happy with this raw backscattering term definition

@callumrollo @vturpin @ptestor do you agree to the new term?