search

NOAA-EMC / GSI

Gridpoint Statistical Interpolation
GNU Lesser General Public License v3.0
66 stars 147 forks source link

RadMon Maintenance Items #100

Closed EdwardSafford-NOAA closed 3 years ago

EdwardSafford-NOAA commented 3 years ago

Several small issues with the RadMon have cropped up. None are major or significant so they will be combined into this ticket. Items include:

EdwardSafford-NOAA commented 3 years ago

Have updated the data extraction and copy functions. Included in this was a change to the low observational counts mechanism. The previous method flagged as "low" an obs count that was < avg - 2 x sdv for a given instrument/channel. That produced a lot of warnings especially for the cris_* instruments. I changed the definition of "low" to < 67% of average and will let this run over the weekend to see if that works out better. First tests are promising.

EdwardSafford-NOAA commented 3 years ago

Have updated the global angle plots. Found that the current master version creates some empty channel plots. That behavior has been corrected. Was able to reduce job submission times and memory.

EdwardSafford-NOAA commented 3 years ago

Committed first phase of updates for regional source. Have DE, IG, and Xfer working for nam. Have updated nam web site to reflect current sat/instruments. Have begun generalizing clean_tankdir.sh to work with both glb and rgn sources.

EdwardSafford-NOAA commented 3 years ago

Moved to hera to test there and found the executables no longer build using the package level (RadMon) cmake build script. Per Mike the build scripts have to change to support hpc-stack libraries on hera (and I guess soon to be everywhere). Things were updated to allow the GSI, enkf, etc to build on hera with those new libraries. I figured out what needed to change in my RadMon script and eventually got it working. I'll need to do that to the OznMon and ConMon but I'll open a separate ticket for those.

EdwardSafford-NOAA commented 3 years ago

Hit a problem on hera on 2/26. I had deleted most of my test data to reduce space. When trying to move new test data for the RadMon using my new RSA SecureID app my permission is denied. I tried the 8 digit numbers with and without my PIN, neither worked. Opened a ticket with the hera help desk. Situation is unchanged as of 3/1 AM.

EdwardSafford-NOAA commented 3 years ago

Completed testing on hera, wcoss_c, wcoss_d. Will wait for pr 110 (hpc-stack libs into cmake builds) before issuing a PR for this.

UPDATE: All testing is going to have to be rerun. Runtime libraries apparently changed on at least wcoss_d, necessitating a change to the RadMon_config file (which loads the modules for the given platform).

EdwardSafford-NOAA commented 3 years ago

Completed retesting. Found I'd missed adding the clean_tankdir.sh to the copy operation to automatically remove the old data directories in TANKverf. Things are ready to go but I'm going to hold short and wait on PR 110 before I issue a PR. Integration with that will require verifying the RadMon build script works correctly on all 3 platforms.