NOAA-EMC / RDASApp

Regional DAS
GNU Lesser General Public License v2.1
1 stars 8 forks source link

Update all submodules #109

Closed hu5970 closed 1 week ago

hu5970 commented 1 month ago

Update all submodules based on: https://github.com/JCSDA/jedi-bundle

hu5970 commented 1 month ago

Sync with JCSDA/jedi-bundle: 1) sync the submodules: fv3 fv3-jedi fv3-jedi-lm femps gsibec ioda mpas-jedi oops saber ufo vader 2) add fv3-interface.cmake 3) remove submodule fms because it needs to be load through model 4) change compile setups for new fv3 model 5) Keep submodule for crtm as is for this issue. Will update it later.

Tested on Hera, Jet and Orion.

hu5970 commented 1 month ago

Tested the RRFS cases on Hera, Jet:

The RDAS can be compiled on JET successfully. The test for RRFS cases shows:

The following tests passed:
    rrfs_fv3jedi_hyb_2022052619
    rrfs_fv3jedi_letkf_2022052619
    rrfs_mpasjedi_2022052619_Ens3Dvar
    rrfs_mpasjedi_2022052619_letkf

80% tests passed, 1 tests failed out of 5

Label Time Summary:
mpi            = 400.01 sec*proc (5 tests)
rdas-bundle    = 400.01 sec*proc (5 tests)
script         = 400.01 sec*proc (5 tests)

Total Test time (real) = 403.44 sec

The following tests FAILED:
    2156 - rrfs_mpasjedi_2022052619_atms_npp_qc (Failed)
Errors while running CTest
Output from these tests are in: /mnt/lfs4/BMC/rtwbl/mhu/jedi/RDASApp/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

The reason for failure is because of the location of CRTM coefficient, The same error exists in current test cases too:

1996:  CRTM_SpcCoeff_Load(FAILURE) : Error reading SpcCoeff file #1, Data/crtm_2.4.0/atms_npp.SpcCoeff.bin
1996:  ufo_radiancecrtm_setup error: failed CRTM_Load_SpcCoeff

The RDAS can be compiled on Hera successfully. The test for rrfs cases shows::

The following tests passed:
    rrfs_fv3jedi_hyb_2022052619
    rrfs_fv3jedi_letkf_2022052619
    rrfs_mpasjedi_2022052619_Ens3Dvar
    rrfs_mpasjedi_2022052619_atms_npp_qc
    rrfs_mpasjedi_2022052619_letkf

100% tests passed, 0 tests failed out of 5

Label Time Summary:
mpi            = 272.21 sec*proc (5 tests)
rdas-bundle    = 272.21 sec*proc (5 tests)
script         = 272.21 sec*proc (5 tests)

Total Test time (real) = 272.70 sec
hu5970 commented 1 month ago

To run ctest, need to load the modules for compiling and set:

setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${RDAS system directory}/build/fv3

hu5970 commented 1 month ago

The test on Orion:

I tried three times (continue) and finally got RDAS compiled successfully. Seems Orion has a policy to punish the big usage of the front node. JEDI clearly used too much resources during compiling. Here is the email I got:

New violation of usage policy by minghu

The ctest cases on Orion are all success:

The following tests passed:
    rrfs_fv3jedi_hyb_2022052619
    rrfs_fv3jedi_letkf_2022052619
    rrfs_mpasjedi_2022052619_Ens3Dvar
    rrfs_mpasjedi_2022052619_atms_npp_qc
    rrfs_mpasjedi_2022052619_letkf

100% tests passed, 0 tests failed out of 5

Label Time Summary:
mpi            = 449.39 sec*proc (5 tests)
rdas-bundle    = 449.39 sec*proc (5 tests)
script         = 449.39 sec*proc (5 tests)

Total Test time (real) = 450.16 sec
SamuelDegelia-NOAA commented 2 weeks ago

I mentioned this in #121 but we can further update the saber module once saber PR #928 is merged.

guoqing-noaa commented 1 week ago

This is complete. When the saber PR is ready, Sam can create a separate issue.