Open vturpin opened 2 years ago
Two parts:
I think it should be interchangeable. If the same method was applied for the whole dataset, why not use a global attribute one time? But if defined in the variable level, it would have priority. Let's consider a standard glider with a standard payload on the California Underwater Glider Network. I could define the standard QC procedure for the CUGN, and mention it only once, at the global level. One day, I do a special mission and add one extra sensor, I could still use the global definition, but for that specific variable, I would define an alternative approach. I think that this concept could be applied in other places. Actually, CF recommends exactly that principle.
The second point is that I don't think that QC should be mandatory but a priority. Ideally, the data provider should do the QC, but if they can't for any reason, we should allow the data to flow, be shared, and address the missing QC down the pipeline. I'm afraid of the situation of a small group collecting some data, willing to share, but not able to do it due to the lack of technical resources to implement such QC. I think we should guide the community on what would be ideal, but we should not impose it as a hard path.
Ok for both locations if this is a recommendation from Cf. But we need to explain that on the document. Let's put it as highly desirable for RTQC method and doi. QC is part of the role of the Data assembly centers as well as converting raw into OG1.0. DAC should have the ressources for RTQC. Isn't there a risk to finally have non QC data sets ?
Yes, there is a risk of having non-QCed data, but in my opinion, that is worse than having no data at all. Once a dataset gets into the system, somebody else (not ideal but) could apply the QC. From the user's perspective, the lack of QC method and flags tells enough how much that data should be trusted.
I do think QCing is important and should be a priority, but I'm afraid of making a harder entry point for new glider operators. I think highly desirable would be appropriate, no less than that. Any group with the required resources would certainly put effort into the high-priority components.
Could you open a PR with such changes on the document, please?
Can we put a level of requierment to a variable attribute?
OG 1.0 meeting notes: Global level RTQC is useful but would need to be a list of methods applied within the data Variable level is needed by DACs. Tool (software, in history attribute for ACDD), thresholds applied (history attribute for ACDD), and procedure (e.g. QUARTOD DOI) both need to be documented. Needs to be downgraded to highly desirable in PG 1.0 (Gui working on pull request). Needs more time and discussion and scoping in OG 1.1.
"RTQC methods" and "RTQC methods doi" are currently mandatory in the global attribute and in the variable attribute. I suggest we remove it from the global attribute and leave it mandatory in the variable attribute.