search

MetOffice / opsinputs

JEDI library generating VarObs and Cx files
BSD 3-Clause "New" or "Revised" License
4 stars 0 forks source link

Add cx namelist for global surface #152

Closed ldhawknesssmith closed 1 year ago

ldhawknesssmith commented 1 year ago

Add CX namelist for global Surface observations. This differs from the OPS equivalent as some fields used in OPS are not for assimilation, but only for mergeback, therefore this namelist removes a number of items which are requested by default. Those fields not required for assimilation have not been made available in the ops-um-interface.

Fixes Issue #153

ctgh commented 1 year ago

The namelist looks fine to me, but I think it would be good to add an extra unit test here to check it's working as expected. You can do that as follows:

(1) Edit opsinputs/test/generate_unittest_netcdfs.py. Near the bottom, there are calls to the routine output_full_cx_to_netcdf for different data types. You can add an entry for Surface there and tailor it to use the correct CX columns.

(2) Create a new file called opsinputs/test/testinput/cxwriter_globalnamelist_surface.yaml (probably easiest to copy from an existing one). Again, modify it to test the correct CX columns.

ldhawknesssmith commented 1 year ago

I have included the test. Thanks @ctgh for the clear instructions for this.

DJDavies2 commented 1 year ago

This seems to be failing on the Cray for me for various lifric-lite-jedi ctests. For example, test_lfriclite_global_interpolator_cubedsphere fails like this:

6: Interpolator created: 6: GlobalInterpolator 6: Test "generic/GlobalInterpolator/testInterpolator" failed with unhandled eckit::Exception: ConfigurationNotFound: [tile index] @  6: Stack trace: backtrace [1] stack has 17 addresses 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/libeckit.so+eckit::BackTrace::dump[abi:cxx11]())0x1b0 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/libeckit.so+eckit::Exception::Exception())0x17b 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/libeckit.so+eckit::Configuration::getIntVector(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&) const)0x5a 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/liblfriclite.so+lfriclite::LocalInterpolator::getTargetJacobians(std::vector<bool, std::allocator >) const)0x733 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/liblfriclite.so+lfriclite::LocalInterpolator::postProcessFields(atlas::FieldSet&, std::vector<bool, std::allocator > const&) const)0x118 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/liboops.so+oops::AtlasInterpolator::apply(oops::Variables const&, atlas::FieldSet const&, std::vector<bool, std::allocator > const&, std::vector<double, std::allocator >&) const)0x67c 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/liboops.so+oops::AtlasInterpolator::apply(oops::Variables const&, atlas::FieldSet const&, std::vector<double, std::allocator >&) const)0x50 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lib/liboops.so+oops::Run::execute(oops::Application const&, eckit::mpi::Comm const&))0xfd 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere) 6: (/lib64/libc.so.6+libc_start_main)0xe6 6: (/home/d03/frwd/cylc-run/opsinputs-152/share/build-mo-cray_intel/lfric-lite-jedi/test/test_lfriclite_global_interpolator_cubedsphere)

Spice tests all seem to pass.

ldhawknesssmith commented 1 year ago

Thanks @DJDavies2 , would we expect changes to the test cx namelist to affect lfric-lite-jedi? I've tried to run the tests myself but cannot get mo-bundle to build successfully at the moment

DJDavies2 commented 1 year ago

I re-ran from a clean build and it succeeded, so perhaps that problem was transitory or due to an incremental build. It seems to work now for me.

ldhawknesssmith commented 1 year ago

Thanks for adding the test! I am happy to approve, but I was wondering if it may be clearer in the long term to explicitly list the fields in Surface.nl, rather than stating which fields are added and which are removed. Doing it the latter way makes verifying the expected upper air and surface fields more difficult. Perhaps something to put in an issue?

Thanks @ctgh .I've created a new issue: https://github.com/MetOffice/opsinputs/issues/155 which I will follow up after this PR is closed.

ldhawknesssmith commented 1 year ago

I re-ran from a clean build and it succeeded, so perhaps that problem was transitory or due to an incremental build. It seems to work now for me.

Thanks @DJDavies2

mikecooke77 commented 1 year ago

@ldhawknesssmith are you happy for me to merge or any last changes?