Open kerfoot opened 1 year ago
This will be going into documentation wiki:
A file named extra_atts.json may be included in a deployment directory to
modify metadata. The top level keys are used to refer to variables, with the exception
of the _global_attrs key, which changes dataset global attributes.
Here is an example where the global attribute institution has been changed,
along with the attributes valid_min and valid_max in the variable longitude:
{
"_global_attrs": {
"institution": "Oregon State University"
},
"longitude": {
"valid_min": -180,
"valid_max": 180
}
}
TODO: Add functionality in Gliders Providers app to allow providers to modify metadata. Determine suitable representation in database for this functionality.
@mdgrossi
Also need a way to alert the DAC administration team when metadata is updated for data sets previously archived by NCEI in order to signal that the NCEI accession record should be updated:
I have created an API end point allowing users to search for accession records by dataset_id, user or glider. The API will be a part of the new status page application.
As things currently stand, all that should be needed for NCEI is a new manifest (i.e., md5) file to accompany the updated data file. That new manifest would be picked up by NCEI and the filename compared to what we already have in the archive. If the data filename is identical to something already in the archive, the appropriate accession will be updated. (If it's a new filename that is not already archived, it will be picked up and archived as usual.)
That said, we should definitely test this to make sure it works as expected. I suggest trying one of the data sets with a misspelled institution. Even though these aren't archived yet, they're in the "staging" area, so the pickup process should be the same. Just let NCEI know when you do this so that we can keep an eye on it to make sure everything works as it ought to.
@benjwadams To clarify: the extra_atts.json file should be placed in the ftp submission directory or the pub erddap location?
As discussed on 2024-02-01 technical tag up, I ran into an issue with a previously archived data set (g652-20230901T1200 in which the data provider used the incorrect WMO id.
My understanding is that the
This will allow for the modification of data set metadata and NCEI archiving under an existing accession number.
Metadata on existing data sets is modified with the use of the extra_atts.json file. The file is created in the following directory:
$PRIV_ERDDAP_ROOT/PROVIDER/DATASET_ID
Here's an example of an extra_atts.json file to change the NC_GLOBAL:id and temperature:standard_name:
A few questions:
This question is related to the following issues: