Test higher dimentional orography handling in convection operators

[jfrost-mo]

This is known to fail currently. Further investigation needed into why it is changing results. The 3D and 4D orography files are just the 2D orography with some manually added coordinates.

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

• [ ] Documentation has been updated to reflect change.
• [x] New code has tests, and affected old tests have been updated.
• [x] All tests and CI checks pass.
• [x] Ensured the pull request title is descriptive.
• [ ] Conda lock files have been updated if dependencies have changed.
• [ ] Attributed any Generative AI, such as GitHub Copilot, used in this PR.
• [x] Marked the PR as ready to review.

[jfrost-mo]

@daflack I'd appreciate your expertise in looking into why results change when the orography has extra dimensions. The 3D and 4D orography is the same data as the 2D orography, but with some added coordinates.

____________________________________________________________________________________ test_inflow_layer_properties_3D_orography _____________________________________________________________________________________

EIB = <iris 'Cube' of m01s20i119 / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>
BLheight = <iris 'Cube' of atmosphere_boundary_layer_thickness / (m) (time: 13; grid_latitude: 600; grid_longitude: 259)>
orography_3D = <iris 'Cube' of surface_altitude / (m) (realization: 1; grid_latitude: 600; grid_longitude: 259)>

    def test_inflow_layer_properties_3D_orography(EIB, BLheight, orography_3D):
        """Reduce a 3D orography filed down to 2D."""
        inflow_layer_properties = convection.inflow_layer_properties(
            EIB, BLheight, orography_3D
        )
        precalculated = iris.load_cube("tests/test_data/convection/ECFlagD.nc")
>       assert np.allclose(inflow_layer_properties.data, precalculated.data)
E       AssertionError: assert False
E        +  where False = <function allclose at 0x2b52c54fca30>(array([[[0., 1., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[1., 1., 1., ..., 0., 0., 0.],\n        [1., 1., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       ...,\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]]]), masked_array(\n  data=[[[0., 1., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[1., 1., 1., ..., 0., 0., 0.],\n         [1., 1., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        ...,\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]]],\n  mask=False,\n  fill_value=1e+20,\n  dtype=float32))
E        +    where <function allclose at 0x2b52c54fca30> = np.allclose
E        +    and   array([[[0., 1., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[1., 1., 1., ..., 0., 0., 0.],\n        [1., 1., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       ...,\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]]]) = <iris 'Cube' of inflow_layer_properties / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>.data
E        +    and   masked_array(\n  data=[[[0., 1., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[1., 1., 1., ..., 0., 0., 0.],\n         [1., 1., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        ...,\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]]],\n  mask=False,\n  fill_value=1e+20,\n  dtype=float32) = <iris 'Cube' of unknown / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>.data

tests/operators/test_convection.py:129: AssertionError
------------------------------------------------------------------------------------------------ Captured log call -------------------------------------------------------------------------------------------------
WARNING  root:convection.py:236 Orography assumed not to vary with time or ensemble member
____________________________________________________________________________________ test_inflow_layer_properties_4D_orography _____________________________________________________________________________________

EIB = <iris 'Cube' of m01s20i119 / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>
BLheight = <iris 'Cube' of atmosphere_boundary_layer_thickness / (m) (time: 13; grid_latitude: 600; grid_longitude: 259)>
orography_4D = <iris 'Cube' of surface_altitude / (m) (time: 1; realization: 1; grid_latitude: 600; grid_longitude: 259)>

    def test_inflow_layer_properties_4D_orography(EIB, BLheight, orography_4D):
        """Reduce a 4D orography filed down to 2D."""
        inflow_layer_properties = convection.inflow_layer_properties(
            EIB, BLheight, orography_4D
        )
        precalculated = iris.load_cube("tests/test_data/convection/ECFlagD.nc")
>       assert np.allclose(inflow_layer_properties.data, precalculated.data)
E       AssertionError: assert False
E        +  where False = <function allclose at 0x2b52c54fca30>(array([[[0., 1., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[1., 1., 1., ..., 0., 0., 0.],\n        [1., 1., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       ...,\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]]]), masked_array(\n  data=[[[0., 1., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[1., 1., 1., ..., 0., 0., 0.],\n         [1., 1., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        ...,\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]]],\n  mask=False,\n  fill_value=1e+20,\n  dtype=float32))
E        +    where <function allclose at 0x2b52c54fca30> = np.allclose
E        +    and   array([[[0., 1., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        [0., 0., 1., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[1., 1., 1., ..., 0., 0., 0.],\n        [1., 1., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       ...,\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]],\n\n       [[0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        ...,\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.],\n        [0., 0., 0., ..., 0., 0., 0.]]]) = <iris 'Cube' of inflow_layer_properties / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>.data
E        +    and   masked_array(\n  data=[[[0., 1., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         [0., 0., 1., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[1., 1., 1., ..., 0., 0., 0.],\n         [1., 1., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        ...,\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]],\n\n        [[0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         ...,\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.],\n         [0., 0., 0., ..., 0., 0., 0.]]],\n  mask=False,\n  fill_value=1e+20,\n  dtype=float32) = <iris 'Cube' of unknown / (unknown) (time: 13; grid_latitude: 600; grid_longitude: 259)>.data

tests/operators/test_convection.py:138: AssertionError
------------------------------------------------------------------------------------------------ Captured log call -------------------------------------------------------------------------------------------------
WARNING  root:convection.py:239 Orography assumed not to vary with time or ensemble member.

[daflack]

@jfrost-mo Initial look through now makes me wonder if is something to do with the masked arrays again? I'll have a more in-depth look on Monday though

[github-actions[bot]]

[jfrost-mo]

The orography file issues have now been fixed (the data was all masked), and there are now separate tests for 3D orography with time or realisation coordinates. Ready to review now.