Closed yunjunz closed 5 months ago
Hi @yuankailiu and @hfattahi, I finally got some time to integrate the ionospheric correction into smallbaselineApp
. Could you take a look at the PR when you get a chance?
I have tested this locally for a stack of alos2 interferograms with ionospheric corrections using the following template options, processed with alosStack, and everything looks fine to me. @yuankailiu could you test it with a stack from topsStack?
mintpy.ionosphericDelay.method = split_spectrum
mintpy.ionosphericDelay.excludeDate12 = 190313-210728,190424-210728
Great! All looks good to me. I've tested from my side with a stack of ionosphere pairs ionStack.h5
processed from stackSentinel.py
.
iono_split_spectrum.py -t inputs/smallbaselineApp_YunjunIon.cfg -f timeseries.h5
where my config is here:
mintpy.ionosphericDelay.method = split_spectrum #[split_spectrum / no], auto for no
mintpy.ionosphericDelay.excludeDate = auto #[20080520,20090817 / no], auto for no
mintpy.ionosphericDelay.excludeDate12 = 20141012-20141024,20220608-20220702 #[20080520_20090817 / no], auto for no
So the script dropped two pairs and run the inversion with no SBAS weights (ifgram_inversion.py ... -w no
). It created the following displacement files: ion.h5
, timeseries_ion.h5
. And the inversion quality files: temporalCoherenceIon.h5
, numInvIon.h5
.
Also, as we chatted, theoretical corrections (external model-based) come before empirical corrections (data-driven). Thus you put iono correction at step after SET and troposphere steps, but before the estimation of ramps and topographic residuals.
It looks like everything works as expected! @yunjunz , thank you for this long waited PR so that we don't need to manually run scripts one-by-one.
A logistical discussion:
For convenience, one thing in the future we can consider is to support importing ionosphere timeseries from stack processor outputs directly? I am not sure. This is just an idea.
At least for topsStack, as what I know, Cunren's code produces a stack of ionosphere pairs and also inverts them in isce2
environment (a simple SBAS inversion without weight just like here). There, we are adviced to plot those pairs with e.g., plotIonPairs.py -idir ion -odir ion_plot
to inspect for the bad pairs and drop them before inversion. The inversion is the same, but the plotting function there is based on mdx, thus much faster than view.py
here.
There is a clear repetative workflow here and there in different environments. Having the whole ionosphere stack imported to Mintpy gives us more flexibility with the whole dataset but the file size of this ionStack.h5 is big like 2GB. Getting the inspection and inversion done in isce2
and just import the timeseries gives me 800MB (well, not a huge reduction because it depends on how dense the original ionosphere network was. So maybe not a big concern in the end).
There is also burstRamp phase estimate of the ionosphere as well. The current stackSentinel
procedure is like the following:
iono_pair --> iono_azshift_pair ----inversion (& inspection)---> iono_azshift_timeseries --> burstRamp_timeseries
Thus the final product are burstRamp timeseries, not burstRamp interferograms. We cannot take them as a burstRamp ifgramStack input into Mintpy consistent as ionStack. So right now I do plotting inspection & manual removal & SBAS inversion in isce2, getting both ionosphere phase and brustRamp timeseries from there, then import these timeseries into MintPy. Anyway, enough logistics...
Thank you @yuankailiu for the testing and discussions. I agree with you. Here are the to-do list I could think of:
diff.py
.load_data.py
into iono_split_spectrum.py
, but am not sure which one is simpler.
Description of proposed changes
This PR integrates the ionospheric correction into the routine workflow of
smallbaselineApp
via a new stepcorrect_ionosphere
. The detailed changes are as below:smallbaselineApp: add a new
correct_ionosphere
step with the following new template options, to support ionospheric correction using split spectrum results fromisce2
stack processingadd
cli/iono_split_spectrum.py
for the cmd interface interactionadd
iono_split_spectrum.py
for the actual workflow of estimating/correcting iono time-series by calling the following individual scripts:add the new script to the entrypoint and sub parsers via changes in the following scripts:
pyproject.toml
src/mintpy/__main__.py
defaults/template.py
: addcorrect_ionosphere
to theSTEP_LIST
Reminders