ESMValGroup / ESMValCore

ESMValCore: A community tool for pre-processing data from Earth system models in CMIP and running analysis scripts.
https://www.esmvaltool.org
Apache License 2.0
42 stars 38 forks source link

Adding fixes for `prc` for ERA5 #2549

Closed malininae closed 1 week ago

malininae commented 1 month ago

I've been looking into CMORization of the prc from ERA5, but I've realized that this variable isn't supported yet. This is the recipe I ran:

# ESMValTool
# recipe_daily_era5.yml
---
documentation:
  description: Create daily ERA5 data

  title: Daily ERA5 data CMORization and creation

  authors:
    - andela_bouwe
    - kalverla_peter
    - alidoost_sarah

  maintainer:
    - unmaintained

  references:
    - acknow_project
    - hersbach20rmets

  projects:
    - ewatercycle

datasets:
  # For the daily diagnostic, always add the next year, otherwise the last day is not cmor compatible
  - {dataset: ERA5, project: native6, type: reanaly, version: v1, tier: 3, start_year: 2022, end_year: 2023}

preprocessors:
  add_one_day: &add_one_day
    extract_time:
      start_year: 2022
      start_month: 1
      start_day: 1
      end_year: 2023
      end_month: 1
      end_day: 1

  daily_mean:
    <<: *add_one_day
    daily_statistics:
      operator: mean

diagnostics:
  daily:
    description: Create daily ERA5 data
    scripts:
      rename:
        script: cmorizers/era5.py
    variables:
      prc:
        mip: E1hr
        era5_name: convective_precipitation
        era5_freq: hourly
        preprocessor: daily_mean

And this is the message I got:

2024-10-11 05:35:38,051 UTC [48997] ERROR   Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_main.py", line 621, in run
    fire.Fire(ESMValTool())
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/fire/core.py", line 143, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/fire/core.py", line 477, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_main.py", line 450, in run
    self._run(recipe, session, cli_config_dir)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_main.py", line 505, in _run
    process_recipe(recipe_file=recipe, session=session)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_main.py", line 147, in process_recipe
    recipe.run()
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_recipe/recipe.py", line 1156, in run
    self.tasks.run(max_parallel_tasks=self.session["max_parallel_tasks"])
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_task.py", line 805, in run
    self._run_sequential()
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_task.py", line 816, in _run_sequential
    task.run()
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_task.py", line 282, in run
    input_files.extend(task.run())
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/_task.py", line 287, in run
    self.output_files = self._run(input_files)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 718, in _run
    product.apply(step, self.debug)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 520, in apply
    self.cubes,
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 532, in cubes
    self._cubes = [ds.load() for ds in self.datasets]
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 532, in <listcomp>
    self._cubes = [ds.load() for ds in self.datasets]
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/dataset.py", line 714, in load
    cube = self._load()
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/dataset.py", line 800, in _load
    result = preprocess(
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 419, in preprocess
    _run_preproc_function(
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/preprocessor/__init__.py", line 364, in _run_preproc_function
    return function(items, **kwargs)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/cmor/fix.py", line 204, in fix_metadata
    cube_list = fix.fix_metadata(cube_list)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/cmor/_fixes/native6/era5.py", line 431, in fix_metadata
    self._fix_units(cube)
  File "/fs/homeu2/eccc/crd/ccrn_shr/emr001/ESMValCore/esmvalcore/cmor/_fixes/native6/era5.py", line 419, in _fix_units
    cube.convert_units(self.vardef.units)
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/iris/cube.py", line 1459, in convert_units
    new_data = _lazy.lazy_elementwise(self.lazy_data(), pointwise_convert)
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/iris/_lazy_data.py", line 539, in lazy_elementwise
    dtype = elementwise_op(np.zeros(1, lazy_array.dtype)).dtype
  File "/space/hall5/sitestore/eccc/crd/ccrn/users/emr001/miniconda3/envs/upd_core_climex_212/lib/python3.10/site-packages/cf_units/__init__.py", line 1918, in convert
    raise ValueError(
ValueError: Unable to convert from 'Unit('m')' to 'Unit('kg m-2 s-1')'.

I've added a fix.