area_weighted regridding fails on several observational datasets

[bascrezee]

The preprocessor function regrid with argument scheme = 'area_weighted' fails on several observational datasets. The error thrown is:

File "/iris/experimental/regrid.py", line 680, in regrid_area_weighted_rectilinear_src_and_grid
    raise ValueError("The horizontal grid coordinates of both the source "
ValueError: The horizontal grid coordinates of both the source and grid cubes must have contiguous bounds.

# Version info
In [2]: iris.__version__                                                                                            
Out[2]: '2.2.1dev0'

Please find below the recipe, including one dataset for which everything works fine. This bug is possibly related to: https://github.com/SciTools/iris/issues/2061

Recipe:

# ESMValTool
---
documentation:
  description: |
    Test recipe recipe_bug_area_weighted.yml 

  authors:
    - crezee_bas

################################################
# Define some default parameters using anchors #
################################################
timeperiod: &timeperiod
  start_year: 2009
  end_year: 2010

################################################
################################################
################################################

datasets:
#
### This dataset works fine:
  - { dataset: CDS-SATELLITE-SOIL-MOISTURE, type: sat, project: OBS, mip: Lmon, version: COMBINED-TCDR-v201812.0.0, tier: 3, << : *timeperiod }
#
### The following three datasets suffer from the bug:
  - { dataset: cds-era5-land-monthly, type: reanaly, project: OBS, mip: Lmon, version: 1, tier: 3, <<: *timeperiod }
  - {dataset: eraintland, type: reanaly, project: OBS6, mip: E6hr, version: 1, tier: 3, frequency: 6hr, <<: *timeperiod }
  - { dataset: cds-era5-monthly, type: reanaly, project: OBS, mip: Lmon, version: 1, tier: 3, <<: *timeperiod }
  - { dataset: CDS-UERRA, type: reanaly, project: OBS6, mip: E6hr, version: UERRA-HARMONIE, tier: 3, frequency: 6hr, << : *timeperiod }

preprocessors:
  pp_regrid:
    regrid:
      target_grid: 1x1
      scheme: area_weighted   # 'nearest' works fine for all datasets above

diagnostics:
  mapplots:
    description: mapplots
    variables:
      sm:
        preprocessor: pp_regrid
        mip: Lmon
    scripts:
      null

I will do my best to look further into this, but any help is greatly appreciated!

[bascrezee]

Indeed the wrapping around of the lon coordinate seems to be a problem, as mentioned in the linked issue. --> the problem seems to be in the lat_bnds (!) more info to follow

e.g. for cds-era5-monthly (area_weighted regridding fails):

 lon_bnds =
  -0.125, 0.125,
  0.125, 0.375,

For CDS-SATELLITE-SOIL-MOISTURE (area_weighted regridding works):

 lon_bnds =
  0, 0.25,
  0.25, 0.5,

The original data (RAWOBS) do not have any bounds, they are added during CMORization. I checked in the documentation that the lon_bnds are correct, and indeed they are consistent with the figure showing the tiles. However, a special comment is written that a view in terms of 'point data' is preferred over the 'tiled data': "However, the ECMWF interpolation software does not conserve area integrals, so this visualisation can be misleading. We recommend you conceptualise ERA5 data as in (a)."

@JaroCamphuijsen @bouweandela did you try out the area_weighted regridder on ERA5 already?

[bascrezee]

The problem here is not in ESMValCore, but rather in the CMORization of the data (which is part of ESMValTool). Therefore I will close this issue and create a new one (this afternoon) in the ESMValTool repository.

[bascrezee]

Although I did not check if the issue occurs for ERA5 (I don't have the data at the moment), I would guess it does. Let me first describe the problem.

Regridding with area_weighted fails with this message:

File "/iris/experimental/regrid.py", line 680, in regrid_area_weighted_rectilinear_src_and_grid
    raise ValueError("The horizontal grid coordinates of both the source "
ValueError: The horizontal grid coordinates of both the source and grid cubes must have contiguous bounds.

The problem is in the swapping of latitude in the CMORization. When swapping the latitude, the latitude bounds (2D) are swapped along both axes. This makes the latitude axis non-contiguous. It is therefore important to first swap the latitudes, and then guess the bounds, in order for the bounds to be contiguous. See https://github.com/ESMValGroup/ESMValTool/pull/1338/commits/f55e0a90de05f033e07179c0f474b82f3b1a27bd#diff-7632d35343f0a13f17c6a0b5c8922c62R86 for how I fixed this for ERA-Interim/Land. Maybe @mattiarighi can check as well.

[bouweandela]

@bascrezee Is this issue still open? I just tested with ERA5 data and there it seems to work fine:

# ESMValTool
# recipe_python.yml
---
documentation:
  description: |
    Example recipe that plots the mean precipitation and temperature.

  authors:
    - andela_bouwe
    - righi_mattia

  maintainer:
    - schlund_manuel

  references:
    - acknow_project

  projects:
    - esmval
    - c3s-magic

datasets:
  - {dataset: ERA5, project: native6, tier: 3, version: 1, type: reanaly}

preprocessors:

  preprocessor1:
    regrid:
      target_grid: 1x1
      scheme: area_weighted

diagnostics:

  diagnostic1:
    description: Air temperature and precipitation Python tutorial diagnostic.
    themes:
      - phys
    realms:
      - atmos
    variables:
      tas:
        preprocessor: preprocessor1
        mip: Amon
        start_year: 1990
        end_year: 1990
    scripts:
      script1:
        script: examples/diagnostic.py
        quickplot:
          plot_type: pcolormesh

[bascrezee]

If I remember correctly, this issue has been solved by https://github.com/ESMValGroup/ESMValTool/pull/1471. So can be closed.