Closed hu5970 closed 1 week ago
Sync with JCSDA/jedi-bundle: 1) sync the submodules: fv3 fv3-jedi fv3-jedi-lm femps gsibec ioda mpas-jedi oops saber ufo vader 2) add fv3-interface.cmake 3) remove submodule fms because it needs to be load through model 4) change compile setups for new fv3 model 5) Keep submodule for crtm as is for this issue. Will update it later.
Tested on Hera, Jet and Orion.
Tested the RRFS cases on Hera, Jet:
The RDAS can be compiled on JET successfully. The test for RRFS cases shows:
The following tests passed:
rrfs_fv3jedi_hyb_2022052619
rrfs_fv3jedi_letkf_2022052619
rrfs_mpasjedi_2022052619_Ens3Dvar
rrfs_mpasjedi_2022052619_letkf
80% tests passed, 1 tests failed out of 5
Label Time Summary:
mpi = 400.01 sec*proc (5 tests)
rdas-bundle = 400.01 sec*proc (5 tests)
script = 400.01 sec*proc (5 tests)
Total Test time (real) = 403.44 sec
The following tests FAILED:
2156 - rrfs_mpasjedi_2022052619_atms_npp_qc (Failed)
Errors while running CTest
Output from these tests are in: /mnt/lfs4/BMC/rtwbl/mhu/jedi/RDASApp/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
The reason for failure is because of the location of CRTM coefficient, The same error exists in current test cases too:
1996: CRTM_SpcCoeff_Load(FAILURE) : Error reading SpcCoeff file #1, Data/crtm_2.4.0/atms_npp.SpcCoeff.bin
1996: ufo_radiancecrtm_setup error: failed CRTM_Load_SpcCoeff
The RDAS can be compiled on Hera successfully. The test for rrfs cases shows::
The following tests passed:
rrfs_fv3jedi_hyb_2022052619
rrfs_fv3jedi_letkf_2022052619
rrfs_mpasjedi_2022052619_Ens3Dvar
rrfs_mpasjedi_2022052619_atms_npp_qc
rrfs_mpasjedi_2022052619_letkf
100% tests passed, 0 tests failed out of 5
Label Time Summary:
mpi = 272.21 sec*proc (5 tests)
rdas-bundle = 272.21 sec*proc (5 tests)
script = 272.21 sec*proc (5 tests)
Total Test time (real) = 272.70 sec
To run ctest, need to load the modules for compiling and set:
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:${RDAS system directory}/build/fv3
The test on Orion:
I tried three times (continue) and finally got RDAS compiled successfully. Seems Orion has a policy to punish the big usage of the front node. JEDI clearly used too much resources during compiling. Here is the email I got:
New violation of usage policy by minghu
The ctest cases on Orion are all success:
The following tests passed:
rrfs_fv3jedi_hyb_2022052619
rrfs_fv3jedi_letkf_2022052619
rrfs_mpasjedi_2022052619_Ens3Dvar
rrfs_mpasjedi_2022052619_atms_npp_qc
rrfs_mpasjedi_2022052619_letkf
100% tests passed, 0 tests failed out of 5
Label Time Summary:
mpi = 449.39 sec*proc (5 tests)
rdas-bundle = 449.39 sec*proc (5 tests)
script = 449.39 sec*proc (5 tests)
Total Test time (real) = 450.16 sec
I mentioned this in #121 but we can further update the saber module once saber PR #928 is merged.
This is complete. When the saber PR is ready, Sam can create a separate issue.
Update all submodules based on: https://github.com/JCSDA/jedi-bundle