ERA5 Monthly average on Gadi

[rbeucher]

On Gadi, ERA5 monthly data are stored as one file / month

|-- 2021
|   |-- tp_era5-land_moda_sfc_20210101-20210131.nc
|   |-- tp_era5-land_moda_sfc_20210201-20210228.nc
|   |-- tp_era5-land_moda_sfc_20210301-20210331.nc
|   |-- tp_era5-land_moda_sfc_20210401-20210430.nc
|   |-- tp_era5-land_moda_sfc_20210501-20210531.nc
|   |-- tp_era5-land_moda_sfc_20210601-20210630.nc
|   |-- tp_era5-land_moda_sfc_20210701-20210731.nc
|   |-- tp_era5-land_moda_sfc_20210801-20210831.nc
|   |-- tp_era5-land_moda_sfc_20210901-20210930.nc
|   |-- tp_era5-land_moda_sfc_20211001-20211031.nc
|   |-- tp_era5-land_moda_sfc_20211101-20211130.nc
|   `-- tp_era5-land_moda_sfc_20211201-20211231.nc
|-- 2022
|   |-- tp_era5-land_moda_sfc_20220101-20220131.nc
|   |-- tp_era5-land_moda_sfc_20220201-20220228.nc
|   |-- tp_era5-land_moda_sfc_20220301-20220331.nc
|   |-- tp_era5-land_moda_sfc_20220401-20220430.nc
|   |-- tp_era5-land_moda_sfc_20220501-20220531.nc
|   |-- tp_era5-land_moda_sfc_20220601-20220630.nc
|   |-- tp_era5-land_moda_sfc_20220701-20220731.nc
|   |-- tp_era5-land_moda_sfc_20220801-20220831.nc
|   |-- tp_era5-land_moda_sfc_20220901-20220930.nc
|   |-- tp_era5-land_moda_sfc_20221001-20221031.nc
|   |-- tp_era5-land_moda_sfc_20221101-20221130.nc
|   `-- tp_era5-land_moda_sfc_20221201-20221231.nc

While ESMValTool recipe usually expect one file / one year (with the 12 months averages)

[rbeucher]

Is there a way to concatenate those files? Should we download averages from CDS?

[rbeucher]

Those are the recipes affected:

66, #68

[rbeucher]

https://github.com/ESMValGroup/ESMValTool/issues/3306

[rbeucher]

Data can be concatenated offline with little effort.

#!/bin/python

import iris
from iris.util import equalise_attributes

var = "2t"

for year in range(1959, 2023):
    print(f"Doing year {year} for variable {var}")
    cubes = iris.load(f"*_{year}*")
    equalise_attributes(cubes)
    cube = cubes.concatenate()
    iris.save(cube, f"{var}_era5_moda_sfc_{year}_monthly.nc")

Now I don't know if it is viable long term