JCSDA-internal / ioda-converters

Various converters for getting obs data in and out of IODA
9 stars 4 forks source link

Add table of central wavenumber to iasi converter #1351

Closed smaticka closed 10 months ago

smaticka commented 1 year ago

Description

Rather than pulling central wavenumber per experiment in the skylab yaml, we are adding a static table of central wavenumber values to be exported during the ioda convertion.

Need to add an 'assign method' in ioda-converter yaml for iasi that looks up a table of values.

Issue(s) addressed

Resolves #1313

rmclaren commented 1 year ago

If you use gettab on this BUFR file you get the following mnemonics:

gettab test/testinput/gdas.t12z.mtiasi.tm00.bufr_d | grep WAVE

| WVNM     | 006029 | WAVE NUMBER                                              |
| LOGRCW   | 025076 | LOG-10 OF (TEMPERATURE-RADIANCE CENTRAL WAVENUMBER)      |

but when you run print_queries.x test/testinput/gdas.t12z.mtiasi.tm00.bufr_d you find out these mnemonics are not available in the test dataset...

You can add a global like you tried, but I'm not sure this this is the right answer as I'm guessing these wavenumbers will be different for different IASI files (?). For more flexibility you could read the data using the python API.

smaticka commented 1 year ago

If you use gettab on this BUFR file you get the following mnemonics:

gettab test/testinput/gdas.t12z.mtiasi.tm00.bufr_d | grep WAVE

| WVNM     | 006029 | WAVE NUMBER                                              |
| LOGRCW   | 025076 | LOG-10 OF (TEMPERATURE-RADIANCE CENTRAL WAVENUMBER)      |

but when you run print_queries.x test/testinput/gdas.t12z.mtiasi.tm00.bufr_d you find out these mnemonics are not available in the test dataset...

You can add a global like you tried, but I'm not sure this this is the right answer as I'm guessing these wavenumbers will be different for different IASI files (?). For more flexibility you could read the data using the python API.

Thanks for checking this @rmclaren. The input test data file hasn't been updated yet, but was the next planned task. Just wanted to get the draft created for conversation on the table entry to begin. I'll work on the new file next.

rmclaren commented 1 year ago

Cool. I should probably update the print_queries.x output to show the description information so that you don't need to switch back and forth... Please be aware that the gettab output often does not really reflect the data in the file (lists subsets that don't exists, often does not list subsets actually in the file). print_queries.x will show you whats actually available in the file.

smaticka commented 1 year ago

Cool. I should probably update the print_queries.x output to show the description information so that you don't need to switch back and forth... Please be aware that the gettab output often does not really reflect the data in the file (lists subsets that don't exists, often does not list subsets actually in the file). print_queries.x will show you whats actually available in the file.

Gotcha. that makes sense then b/c WVNM was still in the output of gettab, but it's not in the print_queries.x output.

BenjaminRuston commented 1 year ago

@smaticka this likely will be moved to future work as updates to the converter are needed to make this happen. Should plan this either with Ron as part for bufr2ioda.x,,, or with SteveH so yaml can be used to write MetaData

BenjaminRuston commented 1 year ago

in short we may want to close this

smaticka commented 1 year ago

Closing to reopen later once 2d ability is allowed for MetaData

BenjaminRuston commented 10 months ago

@smaticka yes this is so confusing, but yes this ioda-converter PR can likely close again

we'll use the table of wavenumbers in a ufo instrumentTests ctest with a reduced ioda file