search

isce-framework / isce2

InSAR Scientific Computing Environment version 2
Other
513 stars 253 forks source link

Scalloping in high coherence interferograms #620

Open ehavazli opened 2 years ago

ehavazli commented 2 years ago

Platform: Sentinel-1A Path: 43 Frame: 88 Acq. Dates: 20220908 - 20221002

Both ESD and geometry coregistration methods produce interferograms with strong scalloping effect. I made tests with varying ESD thresholds (0.25 to 1) and multi looking factors (7x19, 3x9 and 11x31). However, the final interferogram always had visible scalloping effects:

3x9 ML factor with ESD threshold 0.85

Screen Shot 2022-11-09 at 9 15 22 AM

7x19 ML factor with ESD threshold 0.25

Screen Shot 2022-11-08 at 11 02 41 AM

@sssangha and @dbekaert tried different methods as well but no success. I was wondering if anyone ran into a similar issue.

@piyushrpt and @hfattahi do you have any comments and/or recommendations?

piyushrpt commented 2 years ago

Here is a short checklist:

  1. Check IPF version differences
  2. Check ESD histogram
  3. Check if SLCs you are using have been deprecated by ESA - https://sar-mpc.eu/disclaimer/?page=5
  4. Check if orbit files for that time period say a manuever was occurring and quality is not NOMINAL
  5. At the end I would check if ionosphere is causing it
cmarshak commented 2 years ago

I am going to chime in because I ran mostly defaults topsapp xml file (copied at end) and asked Sim, David, and Emre for help. They used stack processor, which I wasn't as interested/familiar. Still, can help answer some of these questions.

I also only searched for "Nominal" and thought once I found it once that it precluded other tags from being corrupted. See item 4 - this may is likely the culprit.

  1. Check IPF version differences

How?

  1. Check ESD histogram

Is that rangeMisregistration.png in the ESD folder? rangeMisregistration

  1. Check if SLCs you are using have been deprecated by ESA

I am using asf_search to download the SLCs right before processing - my understanding is that ASF removes deprecated SLCs.

  1. Check if orbit files for that time period say a manuever was occurring and quality is not NOMINAL

I did this - both orbits had mostly nominal tags (when looking at the XML file). However, there are some xml subtrees with DEGRADED-MANOEUVRE in S1A_OPER_AUX_POEORB_OPOD_20220928T081736_V20220907T225942_20220909T005942. 

<OSV>
      <TAI>TAI=2022-09-08T00:10:59.000000</TAI>
      <UTC>UTC=2022-09-08T00:10:22.000000</UTC>
      <UT1>UT1=2022-09-08T00:10:21.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">310754.687034</X>
      <Y unit="m">-5976974.887435</Y>
      <Z unit="m">-3783280.200037</Z>
      <VX unit="m/s">-1491.464039</VX>
      <VY unit="m/s">-4029.237493</VY>
      <VZ unit="m/s">6251.612949</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:09.000000</TAI>
      <UTC>UTC=2022-09-08T00:10:32.000000</UTC>
      <UT1>UT1=2022-09-08T00:10:31.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">295793.758207</X>
      <Y unit="m">-6016921.811173</Y>
      <Z unit="m">-3720552.387394</Z>
      <VX unit="m/s">-1500.677104</VX>
      <VY unit="m/s">-3960.069759</VY>
      <VZ unit="m/s">6293.832559</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:19.000000</TAI>
      <UTC>UTC=2022-09-08T00:10:42.000000</UTC>
      <UT1>UT1=2022-09-08T00:10:41.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">280742.040117</X>
      <Y unit="m">-6056174.745376</Y>
      <Z unit="m">-3657405.898785</Z>
      <VX unit="m/s">-1509.621605</VX>
      <VY unit="m/s">-3890.440935</VY>
      <VZ unit="m/s">6335.347335</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:29.000000</TAI>
      <UTC>UTC=2022-09-08T00:10:52.000000</UTC>
      <UT1>UT1=2022-09-08T00:10:51.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">265602.226909</X>
      <Y unit="m">-6094729.119679</Y>
      <Z unit="m">-3593847.806621</Z>
      <VX unit="m/s">-1518.295849</VX>
      <VY unit="m/s">-3820.359138</VY>
      <VZ unit="m/s">6376.152476</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:39.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:02.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:01.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">250377.029458</X>
      <Y unit="m">-6132580.445177</Y>
      <Z unit="m">-3529885.230930</Z>
      <VX unit="m/s">-1526.698183</VX>
      <VY unit="m/s">-3749.832543</VY>
      <VZ unit="m/s">6416.243260</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:49.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:12.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:11.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">235069.174950</X>
      <Y unit="m">-6169724.314999</Y>
      <Z unit="m">-3465525.338576</Z>
      <VX unit="m/s">-1534.826996</VX>
      <VY unit="m/s">-3678.869379</VY>
      <VZ unit="m/s">6455.615043</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:11:59.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:22.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:21.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">219681.406463</X>
      <Y unit="m">-6206156.404862</Y>
      <Z unit="m">-3400775.342468</Z>
      <VX unit="m/s">-1542.680721</VX>
      <VY unit="m/s">-3607.477935</VY>
      <VZ unit="m/s">6494.263258</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:09.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:32.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:31.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">204216.482542</X>
      <Y unit="m">-6241872.473620</Y>
      <Z unit="m">-3335642.500772</Z>
      <VX unit="m/s">-1550.257833</VX>
      <VY unit="m/s">-3535.666552</VY>
      <VZ unit="m/s">6532.183421</VZ>
      <Quality>NOMINAL</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:19.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:42.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:41.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">188677.176764</X>
      <Y unit="m">-6276868.363811</Y>
      <Z unit="m">-3270134.116097</Z>
      <VX unit="m/s">-1557.556848</VX>
      <VY unit="m/s">-3463.443624</VY>
      <VZ unit="m/s">6569.371126</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:29.000000</TAI>
      <UTC>UTC=2022-09-08T00:11:52.000000</UTC>
      <UT1>UT1=2022-09-08T00:11:51.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">173066.277317</X>
      <Y unit="m">-6311140.002189</Y>
      <Z unit="m">-3204257.534696</Z>
      <VX unit="m/s">-1564.576330</VX>
      <VY unit="m/s">-3390.817601</VY>
      <VZ unit="m/s">6605.822052</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:39.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:02.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:01.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">157386.586556</X>
      <Y unit="m">-6344683.400256</Y>
      <Z unit="m">-3138020.145642</Z>
      <VX unit="m/s">-1571.314882</VX>
      <VY unit="m/s">-3317.796983</VY>
      <VZ unit="m/s">6641.531957</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:49.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:12.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:11.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">141640.920570</X>
      <Y unit="m">-6377494.654768</Y>
      <Z unit="m">-3071429.380003</Z>
      <VX unit="m/s">-1577.771154</VX>
      <VY unit="m/s">-3244.390320</VY>
      <VZ unit="m/s">6676.496684</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:12:59.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:22.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:21.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">125832.108738</X>
      <Y unit="m">-6409569.948244</Y>
      <Z unit="m">-3004492.710004</Z>
      <VX unit="m/s">-1583.943838</VX>
      <VY unit="m/s">-3170.606213</VY>
      <VZ unit="m/s">6710.712158</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:13:09.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:32.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:31.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">109962.993287</X>
      <Y unit="m">-6440905.549452</Y>
      <Z unit="m">-2937217.648192</Z>
      <VX unit="m/s">-1589.831671</VX>
      <VY unit="m/s">-3096.453310</VY>
      <VZ unit="m/s">6744.174389</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:13:19.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:42.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:41.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">94036.428849</X>
      <Y unit="m">-6471497.813887</Y>
      <Z unit="m">-2869611.746586</Z>
      <VX unit="m/s">-1595.433436</VX>
      <VY unit="m/s">-3021.940310</VY>
      <VZ unit="m/s">6776.879473</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>
    <OSV>
      <TAI>TAI=2022-09-08T00:13:29.000000</TAI>
      <UTC>UTC=2022-09-08T00:12:52.000000</UTC>
      <UT1>UT1=2022-09-08T00:12:51.989957</UT1>
      <Absolute_Orbit>+44905</Absolute_Orbit>
      <X unit="m">78055.282009</X>
      <Y unit="m">-6501343.184239</Y>
      <Z unit="m">-2801682.595829</Z>
      <VX unit="m/s">-1600.747959</VX>
      <VY unit="m/s">-2947.075953</VY>
      <VZ unit="m/s">6808.823589</VZ>
      <Quality>DEGRADED-MANOEUVRE</Quality>
    </OSV>

At the end I would check if ionosphere is causing it

I do not think ionosphere correction is helping the scalloping. I can share a GUNW link if needed.

<?xml version="1.0" encoding="UTF-8"?>
<topsApp>
  <component name="topsinsar">
    <component name="reference">
      <property name="orbit directory">orbits</property>
      <property name="auxiliary data directory">aux_cal</property>
      <property name="output directory">reference</property>
      <property name="safe">['S1A_IW_SLC__1SDV_20221002T151520_20221002T151543_045265_056931_E517.zip']</property>
    </component>
    <component name="secondary">
      <property name="orbit directory">orbits</property>
      <property name="auxiliary data directory">aux_cal</property>
      <property name="output directory">secondary</property>
      <property name="safe">['S1A_IW_SLC__1SDV_20220908T151520_20220908T151542_044915_055D68_78EC.zip']</property>
    </component>
    <property name="swaths">[1, 2, 3]</property>
    <property name="range looks">19</property>
    <property name="azimuth looks">7</property>
    <property name="filter strength">0.5</property>
    <property name="region of interest">[27.223580323190934, 28.982271156038454, 40.761303, 43.569489]</property>
    <property name="demFilename">full_res.dem.wgs84</property>
    <property name="geocodeDemFilename">low_res.dem.wgs84</property>
    <property name="do unwrap">True</property>
    <property name="unwrapper name">snaphu_mcf</property>
    <property name="do ionosphere correction">True</property>
    <property name="apply ionosphere correction">False</property>
    <property name="do ESD">True</property>
    <property name="ESD coherence threshold">0.5</property>
    <property name="use virtual files">True</property>
    <property name="geocode list">['merged/phsig.cor', 'merged/filt_topophase.unw', 'merged/los.rdr', 'merged/topophase.flat', 'merged/filt_topophase.flat', 'merged/filt_topophase_2stage.unw', 'merged/topophase.cor', 'merged/filt_topophase.unw.conncomp', 'merged/topophase.ion']</property>
  </component>
</topsApp>
piyushrpt commented 2 years ago
  1. Check the IW*.xml files or manifest.safe
  2. The other one. This one is for range. The other one is for azimuth
  3. ESA labels some as deprecated but doesn't remove them. Unfortunately, you will have to scan the PDF file in the disclaimers to find this
  4. If the state vectors corresponding to the time of the image come with degraded quality flags - you can't do much. Check state vectors corresponding to the time of the image. Usually this gets captured when first reading the products - https://github.com/isce-framework/isce2/blob/bb4d3b545b0400a9c15e5dd7b6f1cb577f77bfa2/components/isceobj/Sensor/TOPS/Sentinel1.py#L865
  5. I think you will have to apply ionosphere correction to actually subtract it. Else it only estimates it
cmarshak commented 2 years ago

In regards to 4, I don't see any warning WARNING: State Vector at time in my logs (isce.log or the standard error or out).

And searching 15:14:, 15:15:, 15:16: in the potentially bad S1 orbit file had only nominal tags. 🤷 based on the SLC had ID S1A_IW_SLC__1SDV_20220908T151520_20220908T151542_044915_055D68_78EC.

ehavazli commented 2 years ago

  1. IPF versions are 3.52 for both SLCs

  2. The histogram is centered around -0.003

    Screen Shot 2022-11-09 at 1 14 54 PM

  3. Everything is marked as Passed

piyushrpt commented 2 years ago

Those look fine. If its not ionosphere, I cannot think of anything else systematic that could cause this

EJFielding commented 1 year ago

The systematic azimuth shift could be caused by a strong ionospheric gradient on one of the dates. Did you try running the range split-spectrum estimation of the ionospheric effects?

CunrenLiang commented 1 year ago

It might be caused by ionosphere. It would be helpful to look at the merged ESD interferogram.

ehavazli commented 1 year ago

@EJFielding @CunrenLiang I ran iono with ISCE2 but I am not sure how to interpret these results. I have the unwrapped fine interferograms for both upper and lower below:

ion/20220908_20221002/<upper/lower>/merged/fine.unw :

UPPER

ion-upper

LOWER ion-lower

CunrenLiang commented 1 year ago

these are lower and upper band interferograms used for ionospheric phase calculation using range split-spectrum method. For your purpose, the following parameters in input file should be True:

<property name="do ionosphere correction">False</property>
<property name="apply ionosphere correction">False</property>
<property name="consider burst properties in ionosphere computation">False</property>

Then in the merged folder, you should be able to see interferograms before and after ionospheric correction, as well as phase used for correction.