download daily data from era5

[almarbo]

ERA5, downloading daily data

Good morning,

I am currently working on a new notebook to compute climate-projected extreme indices using daily data. I am basing it on the WP4 notebook available at GitHub link. I am interested in adding a feature that allows for the direct download of daily statistics from Era 5.

The CDS website provides an application for this purpose, and the process is outlined in the CDS API documentation link to documentation (refer to section 3.2). I attempted to adapt the request based on the documentation:

('reanalysis-era5-single-levels',
 {'product_type': 'reanalysis',
  'format': 'netcdf',
  'time': '00:00',
  'variable': '2m_temperature',
  'statistic': 'daily_maximum',
  'frequency': '1-hourly',
  'year': ['1999', '2000', '2001', '2002', '2003'],
  'month': ['01', '02', '03', '04', '05', '06', '07', '08', '09', '10', '11', '12']})

However, when I pass this request as an argument to the download.download_and_transform function, it does not recognize it as valid. The error received is as follows:

Exception: The request you have submitted is not valid. None of the data you have requested is available yet. Please revise the period requested. The latest date available for this dataset is: 2023-12-07 09:00.

The API call to retrieve daily statistics from Era 5 has a slightly different structure than the one to obtain hourly data or monthly averaged data, and should include some extra parameters, as specified in the documentation:

c = cdsapi.Client()
result = c.service(
    "tool.toolbox.orchestrator.workflow",
    params={
        "realm": "user-apps",
        "project": "app-c3s-daily-era5-statistics",
        "version": "master",
        "kwargs": <kwargs_dict>,
        "workflow_name": "application"
    }
)

It seems that this feature might not be included when using the function 'download.download_and_transform'.

FYI @lsangelantoni

Thank you, Albert M.

[malmans2]

Hi @almarbo,

That's a feature of the old toolbox, therefore it's not implemented here. It's very likely that ECMWF developer will implement something similar in the new toolbox (i.e., earthkit), but at the moment it's not available.

You should be able to achieve something similar using the caching of download_and_tranform.

Here is an example:

from c3s_eqc_automatic_quality_control import download

# Parameters
year_start = 1999
year_stop = 1999

# Request
collection_id = "reanalysis-era5-single-levels"
request = {
    "product_type": "reanalysis",
    "variable": "2m_temperature",
    "time": "00:00",
}
requests = download.update_request_date(
    request, start=f"{year_start}-01", stop=f"{year_stop}-12", stringify_dates=True
)

# Function to cache
def compute_daily_maximum(ds):
    return ds.resample(forecast_reference_time="1D").max(keep_attrs=True)

# Downlod and transform
ds = download.download_and_transform(
    collection_id,
    requests,
    transform_func=compute_daily_maximum,
    chunks={"year": 1},
)

(As usual, please develop your analysis using a small amount of data. We will then optimize it and run the final notebook on the whole dataset. The example above is already cached on the WP4 VM)

[malmans2]

Closing this as the snipped above should address your question. Feel free to re-open it though.