search

NOAA-EMC / obsproc

2 stars 7 forks source link

Resolving the generation of T2B uprair dump files on wcoss2 #70

Open ilianagenkova opened 2 years ago

ilianagenkova commented 2 years ago

Two branches were created to generate T2B uprair dump files on wcoss2: bufr-dump: feature/dev.iss49636.TAC2BUFR_upper-air obsproc: feature/dev.iss49636.TAC2BUFR_upper-air but they generate 0 size uprair bufr dump files.

Suggestions: 1)review the cmake configuration flags and library use in the relevant cmake files (i.e. https://github.com/NOAA-EMC/bufr-dump/blob/feature/dev.iss49636.TAC2BUFR_upper-air/sorc/bufr_dupupr.fd/CMakeLists.txt) 2)compile code in debug mode (both bufr-dump and obsproc) 3)add print statements to isolate possible issues with BUFRLIB calls 4)abandon the branches above and start from scratch (from tested wcoss2 obsproc code) 5)redriect log files to /ptmp for more time to study them (change trigger and cycqsyb paths) 6)run code with 002/101 tanks composed of smaller in size messages (and hopefully subsets)

ChristopherHill-NOAA commented 2 years ago

The diagnostic output parameter LOUD carries a default setting of 'ON' (upper-case) within each PBS job script that resides in the obsproc/triggers directory. The scripts residing in the bufr-dump/ush directory were originally designed to provide additional diagnostic output with a setting of LOUD='on' (lower-case). Prior to this discovery, available diagnostic output of bufr-dump test runs failed to yield the source of this issue ...

Following a transfer of code development from WCOSS1 to WCOSS2 (from obsproc_dump to bufr-dump), a single line of the file bufr_edtbfr/edtbfr.f - intended to expand the size of readable BUFR data messages - was found to have been inadvertently excluded from the code transfer. Inclusion of the line of code allowed bufr-dump to port the content of radiosonde data originating in the BUFR format from the data tanks to 'uprair' dump files.

Regarding the provided suggestions: 1) CMakeLists.txt files were configured correctly. 2) From the onset, development code was compiled at the debug mode providing the most detailed information (value of 3), as this is the default setting in the PBS job scripts. 3) The output provided by invocation of the correctly set LOUD parameter yielded sufficient diagnostic information. 4) The suggestion to "abandon the branches above and start from scratch" was completely out of place for this issue, considering that the developed code had run seamlessly on WCOSS1 from December 2018 onward. 5) All log files and data output from test runs were temporarily redirected to 'ptmp' diskspace to increase their longevity. 6) The use of tank data featuring an artificially reduced subset size would have added complexity to the test process, and would not have isolated the source of this issue.

ilianagenkova commented 2 years ago

@ChristopherHill-NOAA , thanks for commenting on the suggestions. It would have been more helpful to clarify in 3. that the correct use of LOUD is: export LOUD=on (not export LOUD=ON)

ChristopherHill-NOAA commented 2 years ago

The correct setting of the LOUD parameter is based on matching definitions within the PBS run script and the affected code script in bufr-dump, obsproc, or prepobs. In the case of bufr-dump, a change of "export LOUD=ON" to "export LOUD=on" in the PBS run script was the simplest fix.