Error caused by differences in ERA5 Data Between CDS and CDS-Beta

Hi Lin,

The ERA5 CDS dataset is scheduled to be replaced by the CDS-Beta dataset starting from 26 September 2024. More information can be found here: (https://forum.ecmwf.int/t/differences-in-era5-pressure-level-data-between-cds-and-cds-beta/5014). When I attempt to use the new CDS-Beta data to run the run.py file, I encountered the following error:

Saving model output to /home/yurgao/tc/tropical_cyclone_risk-main/data/era5/test/ Generating land masks... Computing monthly mean and variance of environmental wind... Traceback (most recent call last): File "/home/yurgao/.conda/envs/tc_risk/lib/python3.12/site-packages/xarray/core/dataset.py", line 1393, in _construct_dataarray variable = self._variables[name]


KeyError: 'time'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/yurgao/tc/tropical_cyclone_risk-main/run.py", line 15, in <module>
    compute.compute_downscaling_inputs()
  File "/home/yurgao/tc/tropical_cyclone_risk-main/util/compute.py", line 27, in compute_downscaling_inputs
    env_wind.gen_wind_mean_cov()
  File "/home/yurgao/tc/tropical_cyclone_risk-main/track/env_wind.py", line 97, in gen_wind_mean_cov
    out = dask.compute(*lazy_results)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yurgao/.conda/envs/tc_risk/lib/python3.12/site-packages/dask/base.py", line 664, in compute
    results = schedule(dsk, keys, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yurgao/tc/tropical_cyclone_risk-main/track/env_wind.py", line 130, in wnd_stat_wrapper
    dts = input.convert_to_datetime(ds_ua, ds_ua['time'].values)
                                           ~~~~~^^^^^^^^
  File "/home/yurgao/.conda/envs/tc_risk/lib/python3.12/site-packages/xarray/core/dataset.py", line 1484, in __getitem__
    return self._construct_dataarray(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yurgao/.conda/envs/tc_risk/lib/python3.12/site-packages/xarray/core/dataset.py", line 1395, in _construct_dataarray
    _, name, variable = _get_virtual_variable(self._variables, name, self.dims)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yurgao/.conda/envs/tc_risk/lib/python3.12/site-packages/xarray/core/dataset.py", line 196, in _get_virtual_variable
    raise KeyError(key)
KeyError: 'time'

It appears that the issue may be due to differences in data output between the old CDS and the CDS-Beta. For more details, you might find this report helpful: (https://forum.ecmwf.int/t/differences-in-era5-pressure-level-data-between-cds-and-cds-beta/5014). Would it be possible for you to make adjustments to the existing model code to better align with the new dataset?

Best, Gao

linjonathan / tropical_cyclone_risk

Error caused by differences in ERA5 Data Between CDS and CDS-Beta #7