Closed fcvdb closed 4 months ago
I copied an example of the corrupted GAS, GRACE and Eumetsat Commercial WMO BUFR files on Orion in:
/work2/noaa/jcsda/fvandenb/data/data_from_nrl_bufr
The commands to test each file are (Assuming ioda-bundle/iodaconv
has been built beforehand in directory <build>
and the PYTHONPATH
set up:
export PYTHONPATH="<build>/lib/python3.10:${PYTHONPATH}"
cd <build>/iodaconv/test
../../bin/gnssro_bufr2ioda.py -q -r 1 -d 2021080606 \
-i /work2/noaa/jcsda/fvandenb/data/data_from_nrl_bufr/gras_mta_d20210806_s010638_r14183_ekmi.bfr \
-o gras_mta_d20210806_s010638_r14183_ekmi.nc4 \
&> gras_mta_d20210806_s010638_r14183_ekmi.out
../../bin/gnssro_bufr2ioda.py -q -r 1 -d 2021080606 \
-i /work2/noaa/jcsda/fvandenb/data/data_from_nrl_bufr/grace_d20210806_s012751_edzw.bfr \
-o grace_d20210806_s012751_edzw.nc4 \
&> grace_d20210806_s012751_edzw.out
../../bin/gnssro_bufr2ioda.py -q -r 1 -d 2021080606 \
-i /work2/noaa/jcsda/fvandenb/data/data_from_nrl_bufr/spi_d20220901_s000300_e001300_iuti.bfr \
-o spi_d20220901_s000300_e001300_iuti.nc4 \
&> spi_d20220901_s000300_e001300_iuti.out
@fcvdb is this getting close, did you want me to look at anything
@fcvdb is there an update on this,, is this completed
This issue was resolved through PR#1430 option to retrieve only bending angle also add sub sat id . Closing for now.
The
ioda-converters/src/gnssro/gnssro_bufr2ioda.py
crashes in some instances of corrupted input WMO BUFR files from GRAS, GRACE and Eumetsat Commercial data. This happens when a variable is missing in the file. For example:The occurences are rare. However, because the
gnssro_bufr2ioda.py
converts by batch of input files (~1000), all the other non-corrupted files are not converted when one of those corrupted files is present in the batch.The ideal solution would be that the converter have the capability to skip the corrupted data while processing the batch. Alternatively, a (fast) function to screen the files integrity prior to run the converter could be applied.