ERA5 on-the-fly CMORizer produces scientifically wrong results for some radiation variables

[schlunma]

Describe the bug

Following https://github.com/ESMValGroup/ESMValTool/issues/3694#issuecomment-2204136683, we (@axel-lauer, @LisaBock and myself) had a more detailed look at the radiation variables produced by the ERA5 on-the-fly CMORizer. Here is a summary of what we found:

rlds (surface_downwelling_longwave_flux_in_air; W m-2)

• ERA5 variable we currently use: surface_thermal_radiation_downwards (J m-2)
• ERA5 variable we should use: mean_surface_downward_long_wave_radiation_flux (W m-2)

These two ERA5 variables are basically identical after accounting for the time accumulation. However, we can skip the time accumulation in our code if we use second variable here.

-> Not a big problem, but this would simplify the code

rlns or rls (surface_net_downward_longwave_flux; W m-2)

Note: these variables are identical. The first is our custom version, the second the CMIP6 version.

• ERA5 variable we currently use for rlns: surface_net_thermal_radiation (J m-2)
• ERA5 variable we currently use for rls: mean_surface_net_long_wave_radiation_flux (W m-2)
• ERA5 variable we should use for both: mean_surface_net_long_wave_radiation_flux (W m-2)

These two ERA5 variables are basically identical after accounting for the time accumulation. However, we can skip the time accumulation in our code if we use second variable here.

-> Not a big problem, but this would simplify the code

rlus (surface_upwelling_longwave_flux_in_air; W m-2)

• ERA5 variable we currently use: mean_surface_net_long_wave_radiation_flux (J m-2)
• ERA5 variable we should use: custom derivation from the net and downwards flux, but this cannot be done in the fix yet (https://github.com/ESMValGroup/ESMValCore/issues/1806)

On the surface, outgoing longwave flux is not the same as the net flux (downwards minus upwards)!

-> This is scientifically wrong!

rsds (surface_downwelling_shortwave_flux_in_air; W m-2)

• ERA5 variable we currently use: surface_solar_radiation_downwards (J m-2)
• ERA5 variable we should use: mean_surface_downward_short_wave_radiation_flux (W m-2)

These two ERA5 variables are basically identical after accounting for the time accumulation. However, we can skip the time accumulation in our code if we use second variable here.

-> Not a big problem, but this would simplify the code

rsns or rss (surface_net_downward_shortwave_flux; W m-2)

Note: these variables are identical. The first is our custom version, the second the CMIP6 version.

• ERA5 variable we currently use for both: surface_net_solar_radiation (J m-2)
• ERA5 variable we should use for both: mean_surface_net_short_wave_radiation_flux (W m-2)

These two ERA5 variables are basically identical after accounting for the time accumulation. However, we can skip the time accumulation in our code if we use second variable here.

-> Not a big problem, but this would simplify the code

rsus (surface_upwelling_shortwave_flux_in_air; W m-2)

• ERA5 variable we currently use: mean_surface_net_short_wave_radiation_flux (J m-2)
• ERA5 variable we should use: custom derivation from the net and downwards flux, but this cannot be done in the fix yet (https://github.com/ESMValGroup/ESMValCore/issues/1806)

On the surface, outgoing shortwave flux is not the same as the net flux (downwards minus upwards)!

-> This is scientifically wrong!

Thus, at least for rlus and rsus, we need to adapt the CMORizer.

[axel-lauer]

I just wanted to add that I agree with all of what Manuel described so nicely. I guess we didn't notice these problems as no recipe in main apart from recipe_climwip_brunner2019_med.yml seems to use these variables from ERA5. But this needs to be fixed soon so we don't end up calculating wrong surface radiative fluxes. Mid- to longterm, I would like to bring back the issue of not being able to derive a variable (e.g. swcre) from variables that also require derivation for ERA5 (e.g. rsut, rsutcs) to our attention: https://github.com/ESMValGroup/ESMValCore/issues/1806. For a number of scientific applications (e.g. looking at cloud properties), this is a big obstacle as ERA5 is used frequently.