OneArgo / ArgoVocabs

A repository for the management of issues related to vocabularies managed by the Argo Data Management Team
7 stars 0 forks source link

RR2 and RD2 need updating following revision of QC flag definitions during ADMT-21 #21

Open matdon17 opened 3 years ago

matdon17 commented 3 years ago

At ADMT-21 the descriptions of real-time and delayed-mode flags was updated to be more comprehensive. These changes will need updating in vocabs RR2 and RD2.

Primary contact for these changes is Annie Wong.

vpaba commented 3 years ago

Should note (c) - which contains further information on flags '5' and '8', also added as part of the December 2020 updates to Reference table 2 in Argo User Manual - be captured in the description of these two concepts as well?

matdon17 commented 3 years ago

I'm not sure that's necessary as its usage rather than a definition. There are many other instances of additional information associated with reference tables were we could include it, but it is liable to be more than trouble than it is worth in the context of maintenance - especially considering there will be usage information in various different documents, particularly around QC.

It does raise an interesting question though as to whether we ought to include some additional information in the collection descriptions to mention the Argo documentation. I'm not convinced this is necessary, but might be worth thinking about.

vpaba commented 3 years ago

@gwemon and @matdon17, table 2 at page 72 of the Argo User Manual (https://archimer.ifremer.fr/doc/00187/29825/80869.pdf) includes the following (code | meaning | comment):

‘ ‘ | FillValue | Empty space in NetCDF file

Shall I go ahead and create an empty CODVAL/CODALT (i.e. Identifier and Alternative Label) for this term? I noticed that (non-Argo) tables C05 and C15 on the NVS each have a concept with empty CODVAL/CODALT.

matdon17 commented 3 years ago

I do not know of any reasons why we wouldn't, although that's not to say there isn't.

Are C05 and C15 externally exposed? I can't see them either through the search or VocPrez interface.

gwemon commented 3 years ago

@matdon17 @vpaba there isn't any precedent in the NVS. C15 and C05 are internal BODC vocabs not published on the NVS. I presume that it would not be possible to change the space value ' ' to something else? Or could this record be omitted from the NVS collection? @alko-k what do you think? can we take the risk to have a uri of the form vocab.nerc.ac.uk/collection/RR2/current/ /?

matdon17 commented 3 years ago

Thinking about it a bit more - this is really just the entry for the NetCDF _FillValue variable attribute. I actually don't think it should appear in the reference table (even in the manual) at all.

vpaba commented 3 years ago

This term is new to the table, i.e. it is part of the most recent edits/additions to the manual. One option is to have CODVAL/CODALT = FillValue, rather than an empty space as @gwemon suggested (vocab.nerc.ac.uk/collection/RR2/current/ / looks very strange! plus, I can't imagine many searching for ' ' specifically). Otherwise, @matdon17 we could ask the relevant editors why it was added in the first place, and whether it should be included in the RR2/RD2 collections.

matdon17 commented 3 years ago

I'm not even sure URLs can have spaces, so I doubt the NVS can support it, and even if it can, it is part of the NetCDF standard and seems redundant. We have other reference tables where there is an acceptable _FillValue

One for the next AVTT?

gwemon commented 3 years ago

@matdon17 @vpaba I agree that it would be best to avoid blank spaces in URLs if we can avoid it.

vpaba commented 3 years ago

Thank you both. I will take this to the next AVTT meeting.

delphinedobler commented 6 months ago

The ' ' QC is not really a QC.

It concerns, at least, multiprofiles files ({wmo}_prof.nc): The N_LEVELS dimension is fixed to the maximum number of vertical levels, all profiles considered. However, when a profile has less vertical levels then this maximum, the additionnal levels (i.e. array indices) are filled with FillValue 99999 for the parameter and with a space ' ' for the parameter_QC. Thus the user manual "Empty space in NetCDF file" can also be understood as "empty vertical level in multiprofile file".

As mentioned by Matt, the space " " is indicated as the _FillValue attribute, so should be quite clear from a machine-2-machine perspective. Example: char PRES_QC(N_PROF, N_LEVELS) ; PRES_QC:long_name = "quality flag" ; PRES_QC:conventions = "Argo reference table 2" ; PRES_QC:_FillValue = " " ;

That said, I don't know if there are other occasions where we can have "empty space" for QC. Maybe in synthetic single profiles as well.

I sometimes also encouter(ed) it in old single core profile files, but in this case, the profile file should be corrected (and it will not pass the latest File Checker if it had to anyway).

I would partly agree with what Matt said on its comment from 2021/02/09 : it's part of the netCDF formalism, so I would not add it to the NVS. However, I would not remove it from the user manual but, preferably, I would add a little bit more explanation to detail when this case can occur.