euroargodev / ArgoNetCDF

Argo NetCDF format and content
Creative Commons Zero v1.0 Universal
6 stars 2 forks source link

STATION/TRAJECTORY _PARAMETER rules review #21

Open MarkIggy opened 1 year ago

MarkIggy commented 1 year ago

Review of the FileChecker rules for STATION_PARAMETERS and TRAJECTORY_PARAMETERS.

Referencing the "Description of Argo GDAC File Checks"

PROPOSED CHANGES: (in bold)

Section 5.3 -- Profile files STATION_PARAMETERS: S1: All valid parameter names. Reference table 3. S2: For each parameter, variable exists in file. S3: For each variable with data for profile N, parameter name included in STATION_PARAMETERS for profile N S4: Core-file / profile #1: PRES and TEMP present S5: No duplicate names in list for a given profile. S6: No blank entries within the sequence of names for profile N. (For example, “PRES”, “ “, “TEMP”). Reject

Section 7.1 -- Trajectory files TRAJECTORY_PARAMETERS
T1: All valid parameter names. Reference table 3. T2: For each parameter, variable exists in file. T3: For each variables (with data), parameter name included in TRAJECTORY_PARAMETERS T4: No duplicate names in list. T5: Blank entries within the sequence of names. (For example, “PRES”, “ “, “TEMP”). T5: No blank entries in the list.

DISCUSSION:

Rule S3, S5, S6: Clarify that the rules for STATION_PARAMETERS is on a profile-by-profile basis.

Rule S6: Blank entries must be allowed in profile files because of the way multiple profiles are recorded within the file. Rule S6 says the blank entries must be at the end of the list. Previously, this was only a warning. Propose changing it to a rejection.

Rule T3: The original rule allowed for the situation where the name of a variable in the file did not need to be in the TRAJECTORY_PARAMETERS list if all of its values were _FillValue. With this revision: If a variable is in a file, it must be listed in the TRAJECTORY_PARAMETERS variable.

Rule T5: There is no reason to have blank entries in TRAJECTORY_PARAMETERS. So disallow them.

jegilson commented 1 year ago

I agree with implementation of these changes.

ClaudiaAOML commented 1 year ago

I do not agree with S6 resulting in a rejection. That is a cosmetic change (we have one particular float type that has this issue for BR-files, because of the complicated way it reports some data only in NPROF=1 and others only in NPROF=2). It may seem like a simple change (I wish ...). Problem is: we have a lot of more important updates in the pipeline and more to come because of all the BGC updates that were decided at the previous ADMY meeting.

ClaudiaAOML commented 1 year ago

I wanted to add (and hit the wrong keys to insert a line feed): withholding good profile data because of something that looks weird but is not wrong does not seem like a good idea.

ClaudiaAOML commented 1 year ago

If T5 does not currently result in a warning then I recommend to start with a warning.

If a warning is issued now then a transition to reject seems OK, because it seems like a much easier thing to fix than in the profile files.

mscanderbeg commented 1 year ago

If S6 is too hard to accommodate currently, let's keep it as a warning for now and re-evaluate at the ADMT-24 meeting.
Otherwise, I agree with the suggested changes.