iasi2fbf used by Dual Regression retrieval should be turned into an alpha polar2grid.iasi FE

[rayg-ssec]

The Iasi code metahoard.iasi.fbf (aka iasi2fbf) has too many dependencies. Refactoring it into a standalone package is a subset of creating a polar2grid.iasi frontend. iasi2fbf code should be recast as an alpha version of a polar2grid.iasi entry point, even if the module isn't complete enough to include in an upcoming release.

Packaging in the dual regression bundle can be done similarly to the polar2grid.viirs module use in the CSPP package.

This will be done on the cris_rayg_201303 branch since the two instruments are similar - eventually a polar2grid.interferometer common module may be shared between the two - for instance BT slice generation code.

[rayg-ssec]

Code checked into the cris_rayg_201303 branch this morning which includes a functioning beta of a replacement iasi2fbf without all of metahoard's dependencies (database libraries etc). Also added a quality_flag extracted from the Iasi files, and tested in its most-often-used configurations.

[rayg-ssec]

I've installed a version of ShellB3 with polar2grid.iasi 0.1 (alpha test) on it on galactica for you to experiment with.

To use it, try this in your environment: export PATH=/home/rayg/opt/ShellB3-r713-drrtv/bin:$PATH unset PYTHONPATH python -m polar2grid.iasi.iasi2fbf -o 2013 IASI_xxx_1C_M01_20130310021806Z_20130310023021Z_V_T_20130311161136Z # e.g.

Usage should be the same, otherwise. An additional variable, quality_flag, will have 3 values per IFOV. Simplest way to use it is to sum them and use that as a pass/fail (0 for pass, nonzero fail) on the data.

Once you're happy with it, we can focus on packaging it for inclusion in the Iasi retrieval bundle.

It found 20 bad records in the 2013 test dataset you gave me, going from the GQisQualFlag contents. I'd like to understand why the software flags some reasonable-looking spectra as bad - very likely one of the three bands is all that's affected?

on korra

ipython --pylab from keoni.fbf import Workspace W = Workspace('/mnt/galactica/home/rayg/tmp/iasi_test/2013') qf = W.quality_flag[:] qff = sum(qf,axis=1) # condense 3 band flags to 1 print sum(qff!=0) # 20 bad records brex = nonzero(qff)[0] brad = W.radiances[brex] figure(); plot(brad.transpose())

The procedure for making this runtime is pretty straightforward, using source distributions for polar2grid.core and polar2grid.iasi from cris_rayg_201303 branch at https://github.com/davidh-ssec/polar2grid/tree/cris_rayg_201303 , extracted with "python setup.py sdist". The source packages were combined into the stock eumetsat build of SB3 (which includes eugene EPS format reader) available from ftp.ssec.wisc.edu:/pub/shellb3/.

tar xf ShellB3-Linux-x86_64-20120621-r713-core-eumetsat.tar.gz cd ShellB3 unset PYTHONPATH export PATH=$PWD/bin:$PATH python -m easy_install ~/Software/polar2grid.core-1.1.0.tar.gz python -m easy_install ~/Software/polar2grid.iasi-0.1.tar.gz

The rationale for doing iasi2fbf as a polar2grid optional module is that 80% of what's needed for a swath-extracting front-end for polar2grid to display Iasi SDRs was already present in the iasi2fbf code, which I transplanted and got rid of most of the unnecessary dependencies of in the process. polar2grid.iasi isn't yet ready to run as a fully qualified front-end however - and we can omit it from the distribution until a future release. It will have a lot of overlap with CrIS SDR outside of the file format handling.

[djhoese]

With the changes to polar2grid for v2.0 the iasi package went through a few changes. It no longer attempts (or has a placeholder) for a P2G Frontend. Instead it sits in the repository, as a polar2grid package with no other polar2grid dependencies. I have no problem leaving this package in the repository since it may be useful to polar2grid in the future. @rayg-ssec, if you think this may be better in another repository feel free to remove it from the develop branch. There is also a hyperspectral branch that has less modified code (closer to where @rayg-ssec left it) which includes the CrIS branch which is in a similar state. @rayg-ssec, any thoughts?

[rayg-ssec]

I think I should probably

• clone the repo
• trim it down to just Iasi code and rearrange it
• push it to a new repo under my github account as iasi_tools or equivalent

Sound reasonable?

[djhoese]

Sure, go with what you feel. If we need an iasi frontend in the future we can just make the tools a dependency.