Corrections `enabled` appears to create phase discontinuities at burst boundaries

[scottstanie]

I posted initial interferogram in #99 , but repeating after running the with/without test:

The top was run with corrections enabled=True, the default, and the bottom set enabled=False. This discontinuity is across multiple subswaths (heres further to the right):

If you wanted to test, the top and bottom are

/home/staniewi/dev/sweets-testing/bigger-area/interferograms/stitched/20200529_20200604.int /home/staniewi/dev/sweets-testing/bigger-area-no-corrections//interferograms/stitched/20200529_20200604.int

I was using the branch on #99 for this (same as main COMPASS except for s1_geocode_stack.py), with latest developement version of isce3. To generate them, I did sweets --bbox -119.2 34.1 -117 35.5 --start "2020-05-29" --end "2020-06-6" --track 71

[scottstanie]

To answer @hfattahi 's question: I don't think this is at the frame boundary actually.

import s1reader
s1zips = sorted(Path("/home/staniewi/dev/sweets-testing/data").glob("*0529*.zip"))
bursts1 = s1reader.load_bursts(s1zips[0], None, 3)
bursts2 = s1reader.load_bursts(s1zips[1], None, 3)
print(bursts1)
print(bursts2)

[
    Sentinel1BurstSlc(burst_id=t071_151215_iw3),
    Sentinel1BurstSlc(burst_id=t071_151216_iw3),
    Sentinel1BurstSlc(burst_id=t071_151217_iw3),
    Sentinel1BurstSlc(burst_id=t071_151218_iw3),
    Sentinel1BurstSlc(burst_id=t071_151219_iw3),
    Sentinel1BurstSlc(burst_id=t071_151220_iw3),
    Sentinel1BurstSlc(burst_id=t071_151221_iw3),
    Sentinel1BurstSlc(burst_id=t071_151222_iw3),
    Sentinel1BurstSlc(burst_id=t071_151223_iw3)
]

[
    Sentinel1BurstSlc(burst_id=t071_151224_iw3),
    Sentinel1BurstSlc(burst_id=t071_151225_iw3),
    Sentinel1BurstSlc(burst_id=t071_151226_iw3),
    Sentinel1BurstSlc(burst_id=t071_151227_iw3),
    Sentinel1BurstSlc(burst_id=t071_151228_iw3),
    Sentinel1BurstSlc(burst_id=t071_151229_iw3),
    Sentinel1BurstSlc(burst_id=t071_151230_iw3),
    Sentinel1BurstSlc(burst_id=t071_151231_iw3),
    Sentinel1BurstSlc(burst_id=t071_151232_iw3)
]

The top frame has bursts up to 151223. It looks like the worst discontinuity is between 151225 and 151226, and it likely the corrections applied to t071_151226_iw3.

[scottstanie]

oh @hfattahi it turns out I was wrong and your guess was right: that big discontinuity actually is at a frame boundary, but it's just a frame boundary at the 2nd date only (not the first)

In [12]: s1zips = sorted(Path("/home/staniewi/dev/sweets-testing/data").glob("*0604*.zip"))
    ...: bursts1 = s1reader.load_bursts(s1zips[0], None, 3)
    ...: bursts2 = s1reader.load_bursts(s1zips[1], None, 3)
    ...: print(bursts1)
    ...: print(bursts2)
[
    Sentinel1BurstSlc(burst_id=t071_151217_iw3),
    Sentinel1BurstSlc(burst_id=t071_151218_iw3),
    Sentinel1BurstSlc(burst_id=t071_151219_iw3),
    Sentinel1BurstSlc(burst_id=t071_151220_iw3),
    Sentinel1BurstSlc(burst_id=t071_151221_iw3),
    Sentinel1BurstSlc(burst_id=t071_151222_iw3),
    Sentinel1BurstSlc(burst_id=t071_151223_iw3),
    Sentinel1BurstSlc(burst_id=t071_151224_iw3),
    Sentinel1BurstSlc(burst_id=t071_151225_iw3)
]
[
    Sentinel1BurstSlc(burst_id=t071_151226_iw3),
    Sentinel1BurstSlc(burst_id=t071_151227_iw3),
    Sentinel1BurstSlc(burst_id=t071_151228_iw3),
    Sentinel1BurstSlc(burst_id=t071_151229_iw3),
    Sentinel1BurstSlc(burst_id=t071_151230_iw3),
    Sentinel1BurstSlc(burst_id=t071_151231_iw3),
    Sentinel1BurstSlc(burst_id=t071_151232_iw3),
    Sentinel1BurstSlc(burst_id=t071_151233_iw3),
    Sentinel1BurstSlc(burst_id=t071_151234_iw3)
]

from burst number "...25" to "...26" does happen between s1 granules on June 4th

[scottstanie]

Ok i've done the corrections bisection (where i turn off half the corrections at a time until the problem goes away) Here's the 0-corrections version zoomed into a noticeable feature at the frame boundary:

First half: still a problem

correction_luts:
**
        azimuth_fm_rate: true
        bistatic_delay: true
        geometry_steering_doppler: true
**
        ionosphere_tec: false
        solid_earth_tides: false
        static_troposphere: false
        weather_model_troposphere:
          delay_type: wet_dry
          enabled: false

Second halving: still a problem

      correction_luts:
        azimuth_fm_rate: false
**
        bistatic_delay: true
        geometry_steering_doppler: true
**
        ionosphere_tec: false
        solid_earth_tides: false
        static_troposphere: false
        weather_model_troposphere:
          delay_type: wet_dry
          enabled: false

Finally: turning off bistatic_delay works

      correction_luts:
        azimuth_fm_rate: false
        bistatic_delay: false   # Now it's false
**
        geometry_steering_doppler: true
**
        ionosphere_tec: false
        solid_earth_tides: false
        static_troposphere: false
        weather_model_troposphere:
          delay_type: wet_dry
          enabled: false

[hfattahi]

Thanks @scottstanie. This is great. Since you have the data would you please print out the burst.iw2_mid_range for busrsts in both frames?

[scottstanie]

looking at the labelled picture from the first comment, that feature is between ['t071_151225_iw3', 't071_151226_iw3']

the two ranges are 875604.926001518 and 875284.1711161041 on S1A-20200604 and S1B-20200529

bids = ['t071_151225_iw3', 't071_151226_iw3']
safes = sorted(glob.glob("/u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-test-half0/data/*.SAFE"))
orbits = [s1reader.get_orbit_file_from_dir(f, orbit_dir="/home/staniewi/dev/orbits/") for f in safes]
bursts = [s1reader.load_bursts(f, o, 3, burst_ids=bids) for f, o in zip(safes, orbits)]
[b.iw2_mid_range for b in itertools.chain.from_iterable(bursts)]
#  Out[19]: [875604.926001518, 875604.926001518, 875284.1711161041, 875284.1711161041]

the four sentinel granules i've been using for this are

S1A_IW_SLC__1SDV_20200604T135208_20200604T135236_032868_03CE9E_9E2F.SAFE
S1A_IW_SLC__1SDV_20200604T135234_20200604T135301_032868_03CE9E_8DDE.SAFE
S1B_IW_SLC__1SDV_20200529T135121_20200529T135148_021797_0295FD_D9CF.SAFE
S1B_IW_SLC__1SDV_20200529T135145_20200529T135212_021797_0295FD_F25E.SAFE

[scottstanie]

My mistake- the problem was azimuth_fm_rate

that's the result with only the azimuth_fm_rate correction turned on

      correction_luts:
        azimuth_fm_rate: true
        bistatic_delay: false
        geometry_steering_doppler: false
        ionosphere_tec: false
        solid_earth_tides: false
        static_troposphere: false
        weather_model_troposphere:
          delay_type: wet_dry
          enabled: false

[seongsujeong]

@scottstanie Thanks for capturing this. So I think the discontinuity is happening on the frame boundary? Would you mind taking a look at other burst boundaries on this data, to see if you see the phase discontinuities on there as well when azimuth FM mismatch rate is turned on?

[scottstanie]

Sure thing, you can check with sweets if you wanted to browse yourself in jupyter

import sweets.plotting
sweets.plotting.browse_ifgs("/home/staniewi/dev/sweets-testing/bigger-area-test-azfmrate")

it's also in the other bursts it looks like

going from iw3 to iw2 you can still see it:

but iw1 looks much fainter

it's really just this one line, I think where the frame boundary is on one day (though it's shifted by a burst on the other)

[vbrancat]

@scottstanie would you mind retesting this the latest version (main branch) of COMPASS and s1-reader?

[scottstanie]

I think it looks the same, were there any changes to the azimuth fm rate that might have fixed it?

(here's one of the runconfigs that was used for this screenshot)

```yaml runconfig: groups: dynamic_ancillary_file_group: dem_file: /u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-redo/dem.tif tec_file: null weather_model_file: null input_file_group: burst_id: - t071_151226_iw3 orbit_file_path: - /u/aurora-r0/staniewi/dev/orbits/S1A_OPER_AUX_POEORB_OPOD_20210318T194832_V20200603T225942_20200605T005942.EOF safe_file_path: - /u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-redo/data/S1A_IW_SLC__1SDV_20200604T135234_20200604T135301_032868_03CE9E_8DDE.SAFE pge_name_group: pge_name: CSLC_S1_PGE primary_executable: product_type: CSLC_S1 processing: correction_luts: azimuth_spacing: 0.028 enabled: true range_spacing: 120 troposphere: delay_type: wet_dry geo2rdr: lines_per_block: 1000 numiter: 25 threshold: 1.0e-08 geocoding: flatten: true lines_per_block: 1000 x_posting: 5 x_snap: null y_posting: 10 y_snap: null polarization: co-pol rdr2geo: compute_azimuth_angle: true compute_height: false compute_incidence_angle: true compute_latitude: false compute_layover_shadow_mask: true compute_local_incidence_angle: true compute_longitude: false enabled: false extraiter: 10 geocode_metadata_layers: true lines_per_block: 1000 numiter: 25 threshold: 1.0e-08 product_path_group: product_path: /u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-redo/gslcs product_specification_version: 0.1.5 product_version: '0.2' sas_output_file: /u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-redo/gslcs scratch_path: /u/aurora-r0/staniewi/dev/sweets-testing/bigger-area-redo/gslcs/scratch quality_assurance: browse_image: complex_to_real: amplitude enabled: true equalize: false gamma: 0.5 percent_high: 95 percent_low: 0 output_to_json: false perform_qa: true static_ancillary_file_group: burst_database_file: /home/staniewi/.cache/sweets/burst_map_bbox_only.sqlite3 worker: gpu_enabled: false gpu_id: 0 internet_access: false name: cslc_s1_workflow_default ```

[seongsujeong]

Thanks @scottstanie for re-doing the test. We wanted to check if the PR #113 in s1-reader here is related to the frame discontinuity reported here, which does not seem to be the case.