search

JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA
9 stars 4 forks source link

tropics converter update #1355

Closed BenjaminRuston closed 1 year ago

BenjaminRuston commented 1 year ago

Description

this updates the TROPICS converter for L1B files v3 and maintains backward support for v2

Issue(s) addressed

Resolves #1354

Checklist

BenjaminRuston commented 1 year ago

@mjkagnes123 finished the update of the keys for the files version 3.04.x something like that I think,, but the converter now works for these files. Please give it a try you should be able to switch to the branch and give it a shot I just ran it like so:

python3 /discover/nobackup/bruston/work/JCSDA/JEDI/ioda-bundle-intel/iodaconv/src/hdf5/tropics_2ioda.py -i /discover/nobackup/bruston/data_repos/tropics/nasa_disc/20220216/TROPICS01.BRTT.L1B.Orbit03504.V03-04.ST20220216-012443.ET20220216-025955.CT20230303-224239.nc -d 2022021600 -o ./test.nc4
mjkagnes123 commented 1 year ago

I tested the updated tropics_2ioda.py in discover with TROPICS data and the code works well for old version and new version of data set.

smaticka commented 1 year ago

@BenjaminRuston I don't have access to discover. If you copy the file(s) to orion, I can test it.

smaticka commented 1 year ago

UPDATE: After I rebuilt, this was no longer a problem. Ignore below :)

When I run with the example file on Orion, I get an error that looks like possibly the float int problem. This is the error:

Traceback (most recent call last):
  File "/work2/noaa/da/smaticka/ioda_bundle_26june_intel/ioda-bundle/iodaconv/src/hdf5/tropics_2ioda.py", line 27, in <module>
    from pyiodaconv.def_jedi_utils import ioda_int_type, ioda_float_type, epoch
ImportError: cannot import name 'ioda_int_type' from 'pyiodaconv.def_jedi_utils' (/work2/noaa/da/smaticka/ioda_bundle_26june_intel/build/lib/python3.9/pyiodaconv/def_jedi_utils.py)

After running the converter using this command:

python3 /work2/noaa/da/smaticka/ioda_bundle_26june_intel/ioda-bundle/iodaconv/src/hdf5/tropics_2ioda.py -i /work2/noaa/jcsda/bruston/data_repos/tropics/nasa_disc/20220216/TROPICS01.BRTT.L1B.Orbit03504.V03-04.ST20220216-012443.ET20220216-025955.CT20230303-224239.nc -d 2022021600 -o ../ioda/tropics/test.nc4

After the obs team tagup, we added 3 lines of code to /work2/noaa/da/smaticka/ioda_bundle_26june_intel/build/lib/python3.9/pyiodaconv/def_jedi_utils.py to define ioda_int_type ioda_float_type epoch, and then it worked. Not sure why they weren't defined for me, perhaps a build problem.