Open DocOtak opened 3 weeks ago
Some resources for processing_level: https://wiki.pangaea.de/wiki/Processing_levels only the Argo data status and GCMD "Levels" are in NVS.
More resources, we should absolutely support ORCiDs in the creator URL, we could then populate or validate against the ORCiD record using content negotiation to get a json-ld schema.org/Person record: see https://github.com/ORCID/ORCID-Source/blob/main/CONTENT_NEGOTIATION.md
We could define combining rules to put all this information in the global attributes that fully conform to ACDD, but this would likely be one way (update the globals from variables, not the other way around). For example: the global date_modified would be set to the most recent date seen from all the variables that also have date_modified.
Minor comment that the date_metadata_modified might be modified later in time at the global level than at the variable level - ie that one might not be just one-way variable to global.
Processing level: I love having this, since this is a question I’ve had to field several times already. Assuming our submission doesn’t change, would we automatically assume everything is preliminary unless otherwise stated? Any reason not to implement the PANGAEA (or other existing processing levels such as EOSDIS) schema rather than retaining our own? Creator_url: I would definitely be in favor of adding this attribute to store ORCiDs. Anecdotally, I noticed that BCO-DMO requires data submitters to login via ORCid. Project: This seems really important, but also already a cluster in GO-SHIP i.e. discussed with Alison today our definition of noble gases and how that changes based on what the PI measures i.e. sometimes only He and Tr, other times includes more including Ar, Ne, etc. Would be a good idea to get started on piecing these things apart now and I think it will be helpful in a multitude of ways, not just our metadata model! The date_modified and date_metadata_modified seem super useful. I think adding the ‘lineage statement’ might be helpful as well as a way to store any doi’s that are submitted with the dataset.
netCDF allows for a lot more information than exists in exchange files, with the CCHDO documentation metadata extraction project going, eventually we will need a place to put that metadata. For a while now, I have wanted to store the information contained in the "Bob Headers" in a more structured way. The following ACDD attributes, when pushed down from the global to variable level, should enable the creation of "Bob Headers":
processing_level
,comment
,creator_name
,project
,date_modified
,date_metadata_modified
. Further examination of each of these:processing_level
In ACDD the processing level is a freeform string. We should use this to indicate the following status that very roughly correspond to the satellite communities L0 though L4 processing levels :comment
Free text notes, usually these are very short for each parameter. This is the "notes" part of the Bob Headerscreator_name
This is the PI for the parameter in question, we should use array of strings for multiple PIs in our at rest data files. This is the "who" part of the Bob Headers. There is also acreator_url
attribute that we might consider storing ORCiDs in.project
We need a way to tie multiple variables with the same PI/status together, e.g. nutrients are usually 3~5 variables. In the ACDD docs, a program (GO-SHIP) is made up of multiple projects (Total Carbon, pH, Nutrients, CTD, etc..). Variables that have the same project value would be grouped into the "includes" list in the Bob Headers, thecomment
andcreator_name
would need to be the same to avoid ambiguity. There probably is not a single controlled vocabulary for these project names, they would also likely benefit from some coordination with GO-SHIP.date_modified
If the data itself is changed, this would be updated to be the date it was changed in the data file. The merge_fq accessor already updates this.date_metadata_modified
If only the metadata were modified, this attribute would be updated to the date the change was done. The merge_fq accessor already updates this if the print format is different.The only non standard ACDD usage of the above are being at the variable level rather than global, and the possible use of arrays of strings. We could define combining rules to put all this information in the global attributes that fully conform to ACDD, but this would likely be one way (update the globals from variables, not the other way around). For example: the global
date_modified
would be set to the most recent date seen from all the variables that also havedate_modified
.Things this might make possible:
date_modified
attribute. We can even exclude simple metadata updates that didn't change the values used in science.processing_level
attribute for "collected" and thecreator_name
attribute. This can also be done for bottle data with flag 1.