[DATA REQUEST] Add ERA5 intake-esm datastore

[rbeucher]

Description of the data product

ERA5 Replicated Datasets

ERA5 is the fifth generation model reanalysis of the global climate from the European Centre for Medium-range Weather Forecasts (ECMWF). Data at NCI is replicated through the Copernicus Climate Data Store (CDS) and hosted for use by the Australian research community.

This collection consists of monthly and sub-daily products from 1959 to the present time (approx. 3 months in arrears), including the ERA5.1 re-run for years 2000-2006. Access and location details of these products can be found in the linked catalogue entries under the Associated Records section below or on the NCI ERA5 Community site.

Full details of the model and outputs can be found in the ECMWF ERA5 Data Documentation.

Location of the data product on Gadi

/g/data/rt52

Checklist

Add a "x" between the brackets to all that apply

• [X] This data product is stable (unlikely to change substantially or move)
• [X] This data product is of use to the broader community
• [X] This data product is documented: link to documentation
• [X] This data product is licensed under
  License to use Copernicus products
• [X] Those who want to access this data can be added to the project that houses it

[charles-turner-1]

One of the datasets contained within ERA5 is hourly averaged, monthly datasets, eg.

>>> import os
>>> import xarray as xr
>>> PATH = '/g/data/rt52/era5/single-levels/monthly-averaged-by-hour/2t/2000'
>>> os.listdir(PATH)
['2t_era5_mnth_sfc_20000701-20000731.nc',
 '2t_era5_mnth_sfc_20001201-20001231.nc',
...
 '2t_era5_mnth_sfc_20000601-20000630.nc',
 '2t_era5_mnth_sfc_20001101-20001130.nc']
>>> xr.open_mfdataset(f"{PATH}/*.nc").time
 ['2000-01-01T00:00:00.000000000',
  '2000-01-01T01:00:00.000000000',
  '2000-01-01T02:00:00.000000000', 
  ...,
  '2000-01-01T23:00:00.000000000',
  '2000-02-01T00:00:00.000000000',
  ...,
  '2000-12-01T21:00:00.000000000',
  '2000-12-01T22:00:00.000000000',
  '2000-12-01T23:00:00.000000000']

I'm not sure whether we would want to mark this down as monthly or hourly - can see arguments for both.

EDIT: Just realised the original example was completely ambiguous & updated it to be clearer.

[charles-turner-1]

@rbeucher does this look sensible for metadata.yaml to you?

name: era5-rt52
experiment_uuid: 081d3a94-057c-4ad6-b651-703b3ecd1a5f
description: ERA5 fifth generation model reanalysis of global climate from ECMWF
long_description: >-
  ERA5 is the fifth generation model reanalysis of the global climate from the European 
  Centre for Medium-range Weather Forecasts (ECMWF). ERA5.1 is a re-run of ERA5, 
  for the years 2000 to 2006 only, and was produced to improve upon the cold bias
  in the lower stratosphere seen in ERA5 during this period. This dataset includes 
  the hourly data on pressure levels and has been replicated through the ECMWF 
  data archive (MARS).  Full details of the model and outputs can be found in 
  the ECMWF ERA5 Data Documentation.
model:
- era5
frequency:
- 1mon
- 1day
- 1hr
variable:
- 100u
- 100v
- 10si
- 10u
- 10v
- 2d
- 2t
- alnid
- alnip
- aluvd
- aluvp
- anor
- asn
- bfi
- bld
- blh
- cape
- cbh
- cdir
- cdww
- chnk
- ci
- cin
- cl
- cp
- crr
- csf
- csfr
- cvh
- cvl
- dctb
- deg0l
- dl
- dndza
- dndzn
- dwi
- dwps
- dwww
- e
- es
- ewss
- fal
- fdir
- flsr
- fsr
- gwd
- hcc
- hmax
- i10fg
- ie
- iews
- ilspf
- inss
- ishf
- isor
- istl1
- istl2
- istl3
- istl4
- kx
- lai-hv
- lai-lv
- lblt
- lcc
- lgws
- licd
- lict
- lmld
- lmlt
- lsf
- lshf
- lsm
- lsp
- lspf
- lsrr
- lssfr
- ltlt
- magss
- mbld
- mcc
- mcpr
- mcsr
- mdts
- mdww
- megwss
- mer
- metss
- mgwd
- mgws
- mlspf
- mlspr
- mlssr
- mmtss
- mngwss
- mntss
- mp1
- mp2
- mper
- mpts
- mpww
- mror
- msdrswrf
- msdrswrfcs
- msdwlwrf
- msdwlwrfcs
- msdwswrf
- msdwswrfcs
- msdwuvrf
- mser
- msl
- mslhf
- msmr
- msnlwrf
- msnlwrfcs
- msnswrf
- msnswrfcs
- msqs
- msr
- msror
- msshf
- mssror
- mtdwswrf
- mtnlwrf
- mtnlwrfcs
- mtnswrf
- mtnswrfcs
- mtpr
- mvimd
- mwd
- mwd1
- mwd2
- mwd3
- mwp
- mwp1
- mwp2
- mwp3
- nsss
- p1ps
- p1ww
- p2ps
- p2ww
- pev
- phiaw
- phioc
- pp1d
- ptype
- rhoao
- ro
- rsn
- sd
- sdfor
- sdor
- sf
- shts
- shww
- skt
- slhf
- slor
- slt
- smlt
- sp
- src
- sro
- sshf
- ssr
- ssrc
- ssrd
- ssrdc
- ssro
- sst
- stl1
- stl2
- stl3
- stl4
- str
- strc
- strd
- strdc
- swh
- swh1
- swh2
- swh3
- swvl1
- swvl2
- swvl3
- swvl4
- tauoc
- tcc
- tciw
- tclw
- tco3
- tcrw
- tcslw
- tcsw
- tcw
- tcwv
- tisr
- tmax
- totalx
- tp
- tplb
- tplt
- tsn
- tsr
- tsrc
- ttr
- ttrc
- tvh
- tvl
- u10n
- ust
- uvb
- v10n
- viec
- vigd
- vige
- vign
- viiwd
- viiwe
- viiwn
- vike
- viked
- vikee
- viken
- vilwd
- vilwe
- vilwn
- vima
- vimad
- vimae
- viman
- vimat
- vimd
- viozd
- vioze
- viozn
- vipie
- vipile
- vit
- vithe
- vithed
- vithee
- vithen
- vitoe
- vitoed
- vitoee
- vitoen
- viwvd
- viwve
- viwvn
- vst
- wdw
- wind
- wmb
- wsk
- wsp
- wss
- wstar
- z
- zust
nominal_resolution:
- 31km
- 63km
- 0.36˚
- 1˚
version: 
contact: NCI
email: help@nci.org.au
created: 2020-08-28 
reference: 
- https://rmets.onlinelibrary.wiley.com/doi/10.1002/qj.3803
- https://rmets.onlinelibrary.wiley.com/doi/10.1002/qj.4174
- https://www.ecmwf.int/en/elibrary/81149-global-stratospheric-temperature-bias-and-other-stratospheric-aspects-era5-and
- https://www.ecmwf.int/en/elibrary/19911-low-frequency-variability-and-trends-surface-air-temperature-and-humidity-era5-and
- https://www.ecmwf.int/en/publications/search?searc_all_field=era5
license: https://apps.ecmwf.int/datasets/licences/copernicus/
url: https://geonetwork.nci.org.au/geonetwork/srv/eng/catalog.search#/metadata/f2836_1346_3774_9763
parent_experiment: 
related_experiments:
notes: >-
 This dataset contains hourly and monthly averaged data, as well as monthly 
 data averaged by hour (eg January Mean, 00:00-01:00) and monthly data averaged by day
 (same format, different name convention, eg. January Mean, 00:00-01:00). era5-derived
 contains daily data.
keywords:

If so I'll upload to Gadi & drop in the PR.

[rbeucher]

Yes that looks good. Give it a try and let me know.