koldunovn
commented
1 month ago Maybe it is some general problem with precipitation. Here is error from my attempt to run fourcastnet
Closed koldunovn closed 3 weeks ago
koldunovn
commented
1 month ago Maybe it is some general problem with precipitation. Here is error from my attempt to run fourcastnet
gmertes
commented
1 month ago Hi Nikolay,
Is it crucial that your data is in grib2? As a workaround, you can try grib1 output by calling ai-models with --metadata edition=1
koldunovn
commented
1 month ago Thanks a lot, Gert!
This trick works for both models, although fourcastnet produces something that cdo can't convert to netCDF, but at least I have some grib file :)
Just in case, this is what I have to do to install ai-models with graphcast on our machine.
mamba create -n ai-models3 python=3.10 -y
conda activate ai-models3
pip install --upgrade "jax[cuda12]==0.4.23"
mamba install cudnn=8.9.1.23
pip install git+https://github.com/deepmind/graphcast.git
pip install --upgrade "jax[cuda12]==0.4.23" # it looks it tries to update jax version to more recent, to make sure it's 0.4.23
pip install git+https://github.com/ecmwf-lab/ai-models.git#egg=ai-models
pip install ai-models-graphcast
pip install "numpy<2.0"The biggest problems are:
Thanks a lot once again!
When trying to do a rollout with graphcast, at the data saving stage I get eccodes error:
ERROR
``` 2024-10-06 13:13:59,492 INFO Doing full rollout prediction in JAX: 1 minute 20 seconds. 2024-10-06 13:13:59,493 INFO Converting output xarray to GRIB and saving ECCODES ERROR : concept: no match for paramId=228 ECCODES ERROR : concept: input handle edition=2, centre=ecmf ECCODES ERROR : concept: input handle dataset=era ECCODES ERROR : Please check the Parameter Database 'https://codes.ecmwf.int/grib/param-db/?id=228' 2024-10-06 13:14:13,232 ERROR Error setting edition=2 2024-10-06 13:14:13,232 ERROR Concept no match Traceback (most recent call last): File "/work/ab0995/a270088/mambaforge/envs/ai-models3/lib/python3.10/site-packages/earthkit/data/utils/message.py", line 273, in set return eccodes.codes_set(self._handle, name, value) File "/work/ab0995/a270088/mambaforge/envs/ai-models3/lib/python3.10/site-packages/gribapi/gribapi.py", line 2123, in grib_set grib_set_long(msgid, key, value) File "/work/ab0995/a270088/mambaforge/envs/ai-models3/lib/python3.10/site-packages/gribapi/gribapi.py", line 989, in grib_set_long GRIB_CHECK(lib.grib_set_long(h, key.encode(ENC), value)) File "/work/ab0995/a270088/mambaforge/envs/ai-models3/lib/python3.10/site-packages/gribapi/gribapi.py", line 226, in GRIB_CHECK errors.raise_grib_error(errid) File "/work/ab0995/a270088/mambaforge/envs/ai-models3/lib/python3.10/site-packages/gribapi/errors.py", line 381, in raise_grib_error raise ERROR_MAP[errid](errid) gribapi.errors.ConceptNoMatchError: Concept no match 2024-10-06 13:14:13,237 ERROR Failed to write data to () {'edition': 2, 'generatingProcessIdentifier': 1, 'stream': 'oper', 'expver': 'dmgc', 'class': 'ml', 'template': GribField(tp,None,20230109,1800,6,0), 'startStep': 0, 'endStep': 6} 2024-10-06 13:14:13,237 ERROR grib_set -sedition=2,generatingProcessIdentifier=1,stream=oper,expver=dmgc,class=ml,startStep=0,endStep=6 2024-10-06 13:14:13,237 INFO Saving output data: 13 seconds. 2024-10-06 13:14:13,237 INFO Total time: 2 minutes 9 seconds. Traceback (most recent call last): File "/work/ab0995/a270088/mambaforge/envs/ai-models3/bin/ai-models", line 8, inpackage versions
``` Package Version ------------------------- -------------- absl-py 2.1.0 ai-models 0.7.2 ai-models-graphcast 0.1.0 attrs 24.2.0 cads-api-client 1.4.3 Cartopy 0.23.0 cdsapi 0.7.3 certifi 2024.8.30 cffi 1.17.1 cfgrib 0.9.14.1 cftime 1.6.4 charset-normalizer 3.3.2 chex 0.1.87 click 8.1.7 cloudpickle 3.0.0 colabtools 0.0.1 contourpy 1.3.0 cycler 0.12.1 dask 2024.9.1 dm-haiku 0.0.10 dm-tree 0.1.8 earthkit-data 0.10.6 earthkit-geo 0.2.0 earthkit-meteo 0.1.1 earthkit-regrid 0.3.4 eccodes 2.37.0 ecmwf-api-client 1.6.3 ecmwf-opendata 0.3.10 entrypoints 0.4 etils 1.9.4 filelock 3.16.1 findlibs 0.0.5 flax 0.9.0 fonttools 4.54.1 fsspec 2024.9.0 gcsfs 2024.9.0.post1 google-api-core 2.20.0 google-auth 2.35.0 google-cloud-core 2.4.1 google-cloud-storage 2.18.2 google-crc32c 1.6.0 google-resumable-media 2.7.2 googleapis-common-protos 1.65.0 GPUtil 1.4.0 graphcast 0.1 humanize 4.11.0 idna 3.10 importlib_metadata 8.5.0 importlib_resources 6.4.5 jax 0.4.23 jax-cuda12-pjrt 0.4.23 jax-cuda12-plugin 0.4.23 jaxlib 0.4.23 Jinja2 3.1.4 jmp 0.0.4 jraph 0.0.6.dev0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 kiwisolver 1.4.7 locket 1.0.0 lru-dict 1.3.0 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.9.2 mdurl 0.1.2 ml_dtypes 0.5.0 msgpack 1.1.0 multiurl 0.3.1 nest-asyncio 1.6.0 netCDF4 1.7.1.post2 numpy 1.26.4 nvidia-cublas-cu12 12.6.3.3 nvidia-cuda-cupti-cu12 12.6.80 nvidia-cuda-nvcc-cu12 12.6.77 nvidia-cuda-runtime-cu12 12.6.77 nvidia-cudnn-cu12 9.4.0.58 nvidia-cufft-cu12 11.3.0.4 nvidia-cusolver-cu12 11.7.1.2 nvidia-cusparse-cu12 12.5.4.2 nvidia-nccl-cu12 2.23.4 nvidia-nvjitlink-cu12 12.6.77 opt_einsum 3.4.0 optax 0.2.3 orbax-checkpoint 0.6.4 packaging 24.1 pandas 2.2.3 partd 1.4.2 pdbufr 0.11.0 pillow 10.4.0 pip 24.2 proto-plus 1.24.0 protobuf 5.28.2 pycparser 2.22 Pygments 2.18.0 pyparsing 3.1.4 pyproj 3.7.0 pyshp 2.3.1 python-dateutil 2.9.0.post0 pytz 2024.2 PyYAML 6.0.2 referencing 0.35.1 requests 2.32.3 rich 13.9.2 rpds-py 0.20.0 Rtree 1.3.0 scipy 1.14.1 setuptools 75.1.0 shapely 2.0.6 six 1.16.0 tabulate 0.9.0 tensorstore 0.1.66 toolz 1.0.0 tqdm 4.66.5 trimesh 4.4.9 typing_extensions 4.12.2 tzdata 2024.2 urllib3 2.2.3 wheel 0.44.0 xarray 2024.9.0 zipp 3.20.2 ```originally eccodes was version 2.38.1, downgrading (to 2.37.0) did not help.