Closed bouweandela closed 6 days ago
Potentially related issue: #3694
Note that the recipe ran fine for the previous release: https://esmvaltool.dkrz.de/shared/esmvaltool/v2.10.0/recipe_preprocessor_derive_test_20231212_230320/
I think I know what's going on: https://github.com/ESMValGroup/ESMValCore/pull/2256 changed the way tables are read for derived variables. Before that, toz
was always read from the custom toz
table, which uses DU as units. Now, for CMIP6, it's read from another CMIP6 table that contains it, e.g. AERmon.
Thanks! My fault then ;-) Any idea how to solve this? Could we set the units of the custom CMIP5 toz variable to meters as is done in the CMIP6 cmor table? Or would that cause problems for the diagnostic scripts using these variables?
Yes, I'm 99% percent sure this will break existing diagnostics. This is used by many NCL diagnostics and these are fragile...
We have a convert_units
preprocessor that actually allows this special conversion m <-> DU. We could use that anywhere in our code instead of cube.convert_units
. However, I guess this would require a rc3 for ESMValCore...
An alternative could be to change the units in the custom table and add a convert_units
preprocessor step to the affected recipes. This would probably work, but may expect extra work because an individual preprocessor for toz needs to be used.
However, I guess this would require a rc3 for ESMValCore...
It looks like apart from recipe_preprocessor_derive_test.yml
, only recipe_perfmetrics_CMIP5.yml
and recipe_eyring06jgr.yml
use toz
and both use it with CMIP5 data where this works fine, so maybe we can leave the issue open for this release since no 'real' recipes have been affected.
Sounds like a good idea. So we leave the recipe_preprocessor_derive_test.yml
as it is for now and then update the Core for v2.12?
That sounds easiest. @ehogan What's your opinion?
So leave this in the broken recipe list for v2.11.0 and point to this issue instead of the HDF one? Or have I misunderstood? 🤔
@ehogan in my opinion you can leave it out of that list, it's already been fixed in the current main: https://github.com/ESMValGroup/ESMValCore/pull/2471
@ehogan in my opinion you can leave it out of that list, it's already been fixed in the current main: ESMValGroup/ESMValCore#2471
Wow, that was quick! 😁 The Broken recipe list documentation states "The table is always valid for the latest stable release of ESMValTool.", so I feel we should keep it in the table, since it will be broken for v2.11.0. I would be very happy to remove it as soon as the release is complete! 🤪
Also 100% fine for me 👍
The error is
The units of
toz
are indeed inm
inesmvalcore/cmor/tables/cmip6/Tables/CMIP6_AERmon.json
andesmvalcore/cmor/tables/cmip6/Tables/CMIP6_AERday.json
, but the derive preprocessor produces a cube with units inDU
, which are not convertible tom
according tocf_units
:@schlunma Do you understand what is going wrong here?