Make derivation of total column ozone (`toz`) more flexible and add derivation of stratospheric and tropospheric column ozone

schlunma commented 1 month ago

Description

This PR makes the derivation of total column ozone (toz) more flexible by allowing its derivation from

data that has hybrid levels (e.g., o3 from the AERmon table)
zonal mean data (e.g., o3 from the AERmonZ table)

In addition, this PR adds two derived variables:

stratospheric column ozone (soz), where the "stratosphere" is defined as the region where the O3 mole fraction exceeds 125 ppb
tropospheric column ozone (troz), where the "stratosphere" is defined as the region where the O3 mole fraction is lower than 125 ppb

Backwards-incompatible change

This PR also changes the units of toz in our custom table from DU to m. This is done to make it consistent with the CMIP6 definition. Otherwise, this will lead to various problems when directly comparing CMIP6 output with CMIP5 output (or another project that uses the custom table).

To restore the old behavior, a preprocessor can be added in the recipe to convert m to DU:

convert_units:
  units: DU

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

[x] 🧪 The new functionality is relevant and scientifically sound
[x] 🛠 This pull request has a descriptive title and labels
[x] 🛠 Code is written according to the code quality guidelines
[x] 🧪 and 🛠 Documentation is available
[x] 🛠 Unit tests have been added
[ ] 🛠 ~Changes are backward compatible~ (see above)
[x] 🛠 Any changed dependencies have been added or removed correctly
[x] 🛠 The list of authors is up to date
[x] 🛠 All checks below this pull request were successful

To help with the number pull requests:

🙏 We kindly ask you to review two other open pull requests in this repository

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.83%. Comparing base (2247a29) to head (d7695c2).

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2509 +/- ## ========================================== + Coverage 94.77% 94.83% +0.05% ========================================== Files 249 251 +2 Lines 14081 14179 +98 ========================================== + Hits 13345 13446 +101 + Misses 736 733 -3 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

schlunma commented 1 month ago

This can be tested with the following recipe:

# ESMValTool
---
documentation:
  title: test
  description: test
  authors:
    - schlund_manuel
  maintainer:
    - schlund_manuel

preprocessors:
  du:
    convert_units:
      units: DU

diagnostics:

  test:
    variables:
      toz:
        derive: true
        force_derivation: true
        exp: historical
        start_year: 2013
        end_year: 2014
        preprocessor: du
        additional_datasets:
          - {project: CMIP6, dataset: CESM2-WACCM, ensemble: 'r1i1p1f1', grid: gn, mip: Amon}
          - {project: CMIP6, dataset: GISS-E2-1-G, ensemble: 'r1i1p3f1', grid: gn, mip: AERmon}
      troz:
        derive: true
        force_derivation: true
        exp: historical
        start_year: 2013
        end_year: 2014
        preprocessor: du
        additional_datasets:
          - {project: CMIP6, dataset: CESM2-WACCM, ensemble: 'r1i1p1f1', grid: gn, mip: Amon}
          - {project: CMIP6, dataset: GISS-E2-1-G, ensemble: 'r1i1p3f1', grid: gn, mip: AERmon}
      soz:
        derive: true
        force_derivation: true
        exp: historical
        start_year: 2013
        end_year: 2014
        preprocessor: du
        additional_datasets:
          - {project: CMIP6, dataset: CESM2-WACCM, ensemble: 'r1i1p1f1', grid: gn, mip: Amon}
          - {project: CMIP6, dataset: GISS-E2-1-G, ensemble: 'r1i1p3f1', grid: gn, mip: AERmon}
    scripts:
      null

recipe_test_ozone.yml.txt

schlunma commented 1 month ago

As far as I can tell with, toz and soz look good. However, I get some ugly results for troz, which include lots of artifacts:

grafik

Most likely this is the result of lower pressure levels being masked in the input data (o3):

grafik

valeriupredoi commented 1 month ago

Manu, can you name these islands? islands

schlunma commented 1 month ago

Manu, can you name these islands? ![islands](https://private-user-images.githubusercontent.com/28983971

Not all of them, but Google Maps certainly can 😄

FranziskaWinterstein commented 1 week ago

Hey @schlunma, thank you for taking care of this.

I made my own attempt to build troz.py (see below), by basically calculating troz as troz = toz - soz. This reduces the doubling of code, but introduces dependencies from troz on soz. What do you think about that?

"""Derivation of variable ``soz``."""

import dask.array as da
import iris

from ._baseclass import DerivedVariableBase
from .toz import DerivedVariable as Toz
from .soz import DerivedVariable as Soz

class DerivedVariable(DerivedVariableBase):
    """Derivation of variable ``troz``."""

    @staticmethod
    def required(project):
        """Declare the variables needed for derivation."""
        if project == 'CMIP6':
            required = [{'short_name': 'o3'}, {'short_name': 'ps'}]
        else:
            required = [{'short_name': 'tro3'}, {'short_name': 'ps'}]
        return required

    @staticmethod
    def calculate(cubes):
        """Compute tropspheric column ozone.

        Note
        ----
        In the calculation of ``troz``, the surface air pressure (``ps``) is
        used to determine the pressure level width of the lowest layer.

        The calculation of ``troz`` consists of three steps:
        (1) Use derivation function of ``toz`` to calculate ``toz``.
        (2) Use derivation function of ``soz`` to calculate ``soz`` (using the
            masked data).
        (3) Calculate difference of ``toz`` and ``soz`` to calculate ``troz``.

        """
        o3_cube = cubes.extract_cube(
            iris.Constraint(name='mole_fraction_of_ozone_in_air'))
        ps_cube = cubes.extract_cube(
            iris.Constraint(name='surface_air_pressure'))

        # (1) Use derivation function of toz to calculate toz
        cubes = iris.cube.CubeList([o3_cube, ps_cube])
        cube_toz = Toz.calculate(cubes)

        # (2) Use derivation function of soz to calculate soz
        cube_soz = Soz.calculate(cubes)

        # (3) Calculate troz as difference of toz and soz
        return cube_toz - cube_soz

FranziskaWinterstein commented 1 week ago

I successfully tested the given recipe as well as applied it to the diagnostics to plot hovmoeller_vs_lat_or_lon and map plots from multi_dataset.py. No problems.

FranziskaWinterstein commented 1 week ago

The map plots of the climatology 2013 - 2014 of troz do not show those artefacts. I suppose they only appear without time averaging. If it is data dependent, I guess, there is not much we can do about it? map_CESM2-WACCM_troz Created with: recipe_test_ozone_map.yml.txt

ESMValGroup / ESMValCore