Open adele-morrison opened 1 month ago
@marc-white, the effort in adding these will be in making sure all the required fields are available and populated in each experiment's metadata.yaml
. Some of them look pretty good, some need work and some don't even have a metadata.yaml
...
Creating/populating the metadata.yaml
is really something that the experiment owner (@adele-morrison?) needs to do because they know about the experiment. Once they are up to standard it should be as simple as adding the paths to config/access-om2.yaml
@adele-morrison are you the experiment owner for these, or are you able to prod the owner(s) to update/create a metadata.yaml
for each experiment?
These docs will be useful for knowing what's needed in the metadata.yaml
Where do I find the experiment_uuid
?
With new versions of Payu, this will be autogenerated. But for back-filling old experiments you can just generate one for each experiment by running:
uuidgen
Ok, this might take a while. As a start, I think the metadata.yaml is ready to go for these experiments: 01deg_jra55v13_ryf9091_qian_wthmp 01deg_jra55v13_ryf9091_qian_wthp 01deg_jra55v13_ryf9091_easterlies_down10 01deg_jra55v13_ryf9091_easterlies_up10 01deg_jra55v13_ryf9091_easterlies_up10_meridional 01deg_jra55v13_ryf9091_easterlies_up10_zonal 01deg_jra55v13_ryf9091_weddell_down2 01deg_jra55v13_ryf9091_weddell_up1
The metadata.yaml looks good for these ones too, but there are permission issues with some of the directories:
01deg_jra55_ryf_Control
01deg_jra55_ryf_ENFull
01deg_jra55_ryf_LNFull
@hrsdawson could you please create metadata.yaml files for these experiments on cj50: 01deg_jra55v13_ryf9091_ssp245 01deg_jra55v13_ryf9091_ssp245_mw 01deg_jra55v13_ryf9091_ssp585
Is there supposed to be the 01deg_jra55v13_ryf9091_ssp585_mw experiment on cj50 also? I can't see it.
@kialstewart, currently the metadata.yaml for these experiments are all identical: 01deg_jra55_SAMextr_0203_from9091 01deg_jra55_SAMextr_1011_from9091 01deg_jra55_SAMextr_9192_from9091 01deg_jra55_SAMextr_9899_from9091
Can you please fix?
@mauricehuguenin I'm still having permission issues accessing the data in these directories on /g/data/cj50/access-om2/raw-output/access-om2-01/:
01deg_jra55_ryf_Control/output984/ - output989 01deg_jra55_ryf_ENFull/output985 01deg_jra55_ryf_LNFull/output984 - output989
I think we'll need to sort this before we can add to the Intake Catalog.
Thanks Adele, I sent help@nci.org.au a new mail again if they could make the folder group-readable. When I tried using chmod u+w
it did not work.
Description of the data product
There are a bunch of COSIMA experiments that are missing in the Intake Catalog. As a start, it would be great to have everything here:
/g/data/cj50/access-om2/raw-output/access-om2-01/
in the Catalog, with the exception of01deg_jra55v13_ryf9091
, which we already have fromik11
.That means, adding these experiments: 01deg_jra55_ryf_Control
01deg_jra55v13_ryf9091_qian_wthmp 01deg_jra55_ryf_ENFull
01deg_jra55v13_ryf9091_qian_wthp 01deg_jra55_ryf_LNFull
01deg_jra55v13_ryf9091_ssp245 01deg_jra55_SAMextr_0203_from9091
01deg_jra55v13_ryf9091_ssp245_mw 01deg_jra55_SAMextr_1011_from9091
01deg_jra55v13_ryf9091_ssp585 01deg_jra55_SAMextr_9192_from9091
01deg_jra55v13_ryf9091_weddell_down2 01deg_jra55_SAMextr_9899_from9091
01deg_jra55v13_ryf9091_weddell_up1
01deg_jra55v13_ryf9091_easterlies_down10
01deg_jra55v13_ryf9091_easterlies_up10
01deg_jra55v13_ryf9091_easterlies_up10_meridional
01deg_jra55v13_ryf9091_easterlies_up10_zonal
Location of the data product on Gadi
/g/data/cj50/access-om2/raw-output/access-om2-01/
Checklist
Add a "x" between the brackets to all that apply