Load GRIB variables with "unknown" typeOfLevel using custom ecCodes tables

[torfsen]

I've posted this question on StackOverflow before but didn't get any answers. Feel free to answer it there and/or close this ticket.

I'm trying to load GRIB2 files from DWD's ICON model using xarray and cfgrib. Most variables work fine, but for some (like for example CLCL), xarray.open_dataset raises the following error:

Traceback (most recent call last):
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1398, in _construct_dataarray
    variable = self._variables[name]
KeyError: 'CLCL'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "DAT-2634/grib_file_loaders.py", line 276, in <module>
    df_cfgrib_sf = read_cfgrib_sf(grib_fns, nodes, params)
  File "DAT-2634/grib_file_loaders.py", line 115, in read_cfgrib_sf
    x = xr.open_dataset(
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1502, in __getitem__
    return self._construct_dataarray(key)
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 1400, in _construct_dataarray
    _, name, variable = _get_virtual_variable(
  File "/shared/conda/envs/devtools/lib/python3.8/site-packages/xarray/core/dataset.py", line 173, in _get_virtual_variable
    ref_var = variables[ref_name]
KeyError: 'CLCL'

when I inspect the file via grib_ls I get

$ grib_ls ICON_europe_reg_0.125x0.125_2021101900_f000.grib2 | grep CLCL
2            edzw         20211019     fc           regular_ll   0            unknown      800          CLCL         grid_simple 

I've read in the issues #195 and #213 that this is probably due to the use of local codes and that I might be able to fix the issue using the appropriate code tables. DWD does provide GRIB tables for ecCodes, but I'm not sure how to use them in combination with cfgrib.

How can I use the custom code tables with cfgrib? Or is there another way to load these variables?

[iainrussell]

Hello @torfsen,

Once you have the tables downloaded and put somewhere on disk, you need to set the environment variable "ECCODES_DEFINITION_PATH" to point to where the new tables are. See the bottom of this page, which also describes how to create the tables (which you do not need to do of course!) https://confluence.ecmwf.int/display/UDOC/Creating+your+own+local+definitions+-+ecCodes+GRIB+FAQ

Cheers, Iain

[torfsen]

Thanks, @iainrussell, that was exactly what I was looking for. Adding the path of the custom code tables (and that of the base tables that come with ecCodes) to ECCODES_DEFINITION_PATH worked wonderfully.

[iainrussell]

Brilliant, thanks for letting me know @torfsen !