ACCESS-NRI / access-nri-intake-catalog

Tools and configuration info used to manage ACCESS-NRI's intake catalogue
https://access-nri-intake-catalog.rtfd.io
Apache License 2.0
8 stars 1 forks source link

[DATA REQUEST] Bureau of Meteorology Atmospheric Regional Projections for Australia (BARPA) #197

Open rbeucher opened 1 day ago

rbeucher commented 1 day ago

Description of the data product

BARPA Projections contains a moderate-resolution and a kilometre-scale regional climate projections over Australia and surrounding regions. The current collections are developed based on CMIP6 global climate projections. It consists of an ensemble of historical and future climate model fields. It can be used for climate research, impact modelling, developing climatology products, and future likelihood products, driving downstream impact modelling (e.g., hydrological, coastal, and fire), for case studies and stress testing of environments, infrastructure and assets under different future scenarios.

Location of the data product on Gadi

NCI intake catalog available in /g/data/py18

Checklist

Add a "x" between the brackets to all that apply

marc-white commented 23 hours ago

Have requested membership of this project to investigate.

marc-white commented 2 hours ago

Existing CMIP6 Intake-ESM catalog, so no new builder should be required. However, there isn't a metadata.yaml for this experiment (https://access-nri-intake-catalog.readthedocs.io/en/latest/management/building.html#metadata-yaml-files).

@rbeucher who would be the best person to add one for this experiment?

rbeucher commented 2 hours ago

I am not sure how @dougiesquire did for the fs38, oi10 etc collection. It should be similar.

marc-white commented 2 hours ago

It's interesting, the config files inside access-nri-intake-catalog reference metadata.yaml for those experiments that don't exist. I'm trying to work out if they're just dummies to keep the system happy, or if they did once exist but now aren't needed for some reason.

dougiesquire commented 2 hours ago

They should exist. If they don't it sounds like they could've been accidentally deleted... I can take a better look later today

marc-white commented 1 hour ago

@dougiesquire the CMIP5 and CMIP6 experiments all refer to yamls stored in /g/data/tm70/intake, which doesn't exist. A search shows that there are a few metadata.yaml hiding within /g/data/tm70, but nothing that access-nri-intake-catalog appears to reference.

dougiesquire commented 1 hour ago

Hmm, that folder used to exist. I guess it's been deleted (possibly by me??).

It should be possible to recreate the contents of the metadata.yaml from the latest (or prior) version of the catalog. E.g. something like:

catalog["cmip6_fs38"].metadata
rbeucher commented 1 hour ago

NCI has changed the way they deploy intake catalogs. They are now in each collection. That might be what is happening here.

marc-white commented 58 minutes ago

I'll shift the missing YAMLs into a new issue. Meanwhile, I think there's enough information in the existing catalog.json to be able to construct a sensible metadata.yaml myself. Should I put it in a 'master' folder under tm70, or drop it in the same place as the catalog? I presume the latter, but I'm nervous about touching data archives without explicit permission.