ecmwf-lab / ai-models-graphcast

Apache License 2.0
57 stars 19 forks source link

cfgrib DatasetBuildError when using ERA5 initialisation via CDS #3

Closed kieranmrhunt closed 10 months ago

kieranmrhunt commented 10 months ago

Trying to run GraphCast as follows: ai-models --input cds --date 20230815 --time 0000 graphcast

Returns the following error indicating a dataset size mismatch, regardless of choice of start date:

Traceback (most recent call last):
  File "/home/users/kieran/miniconda3/envs/ai-models/bin/ai-models", line 8, in <module>
    sys.exit(main())
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/ai_models/__main__.py", line 291, in main
    _main()
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/ai_models/__main__.py", line 264, in _main
    model.run()
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/ai_models_graphcast/model.py", line 199, in run
    training_xarray, time_deltas = create_training_xarray(
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/ai_models_graphcast/input.py", line 60, in create_training_xarray
    fields_sfc.to_xarray().rename(GRIB_TO_XARRAY_SFC).isel(number=0, surface=0)
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/climetlab/readers/grib/xarray.py", line 106, in to_xarray
    result = xr.open_dataset(
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/xarray/backends/api.py", line 573, in open_dataset
    backend_ds = backend.open_dataset(
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/xarray_plugin.py", line 108, in open_dataset
    store = CfGribDataStore(
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/xarray_plugin.py", line 40, in __init__
    self.ds = opener(filename, **backend_kwargs)
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/dataset.py", line 750, in open_fieldset
    return open_from_index(filtered_index, read_keys, time_dims, extra_coords, **kwargs)
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/dataset.py", line 726, in open_from_index
    dimensions, variables, attributes, encoding = build_dataset_components(
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/dataset.py", line 680, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/home/users/kieran/miniconda3/envs/ai-models/lib/python3.10/site-packages/cfgrib/dataset.py", line 611, in dict_merge
    raise DatasetBuildError(
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='time' value=Variable(dimensions=('time',), data=array([1692036000, 1692057600])) new_value=Variable(dimensions=('time',), data=array([1691992800, 1692036000]))

Versions:

ai-models==0.2.15
jax==0.4.20
cfgrib==0.9.10.4
xarray==2023.10.1
kieranmrhunt commented 10 months ago

Fixed by cloning the latest ai-models-graphcast from git.