bufr2ioda details and ongoing issues

[guoqing-noaa]

The ioda_bufr task is to convert all conventional observations (aircraft, ascatw, gpsipw, mesonet, profiler, rassda, satwnd, all surface types, upperair) in prepbufr to ioda files.

The executable is under rrfs-workflow/exec, which is linked from sorc/RDASApp/build/bin/bufr2ioda.x bufr2ioda.x needs a few libraries to work as described in issue #55

bufr2ioda.x can only run in serial but needs 4G memory when converting mesonet obs (per @spanNOAA ). If only one core is requested for the ioda_bufr task, one needs to add this to the slurm job card:
#SBATCH --mem=4GB

Currently, ioda converter failed for the upper air obs:

+ 31 + srun ./bufr2ioda.x prepbufr_upperair.yaml
PRE-MAIN-INFO BufrParser: Parsing file ./prepbufr
PRE-MAIN-INFO Executing Queries
PRE-MAIN-ERROR Exception: BadValue: No valid BUFR subsets were found from your queries! Please make sure you are querying for valid subsets that exist in ./prepbufr. Otherwise there might be a problem with the BUFR file (no subsets). 
terminate called after throwing an instance of 'eckit::BadValue'
  what():  BadValue: No valid BUFR subsets were found from your queries! Please make sure you are querying for valid subsets that exist in ./prepbufr. Otherwise there might be a problem with the BUFR file (no subsets).
srun: error: k368: task 0: Aborted (core dumped)
srun: Terminating StepId=6817126.8
+ 34 + export err=134
+ 34 + err=134
+ 34 + err_chk

[spanNOAA]

Did you used the ioda converter to convert the prepbufr file containing upper air data, which indicates that the prepbufr file is valid either at 12Z or 00Z?

[guoqing-noaa]

Good point. The failed ioda_bufr task is at 03z. Does your upper air data mean "sounding" only?

[spanNOAA]

Good point. The failed ioda_bufr task is at 03z. Does your upper air data mean "sounding" only?

the upper air yaml file is only used to convert ADPUPA.

[guoqing-noaa]

ok, thanks for the clarification! In this sense, we would like to rename it to ADPUPA instead of upper air which is a little bit of confusing.

[spanNOAA]

you can also rename surface to adpsfc.

[guoqing-noaa]

Yes, that reduces confusion. Do we need to make any other name changes?

[spanNOAA]

Yes, that reduces confusion. Do we need to make any other name changes?

Probably no cause others are using their subsets name, except profiler contains both PROFLR and VADWND.

[spanNOAA]

Just hold on for a moment to rename those files, because I'm about to submit a new pull request to includ time offset into the YAML files.

[guoqing-noaa]

ok.

[guoqing-noaa]

When ADPUPA data is not available, the bufr2ioda.x should only print out INFO that no valid BUFR subset was found and should not exit with an error. Because there is some sounding data available at cycles other than 00z, and 12z.