Five-km misregistration for some scenes

[rzinke]

Describe the bug Reprocessing some of the Tibet data seems to have introduced a spatial misregistration of ~5 km. So far, I have found > 60 interferograms where this is a significant issue, each composed of ~10 products (i.e., on the order of 600 products could be affected). Track 41 is the only track where I have definitely identified this issue so far, though it makes the track 41 data unusable because the network is no longer connected if all misregistered scenes are thrown out. Furthermore, other tracks could be affected and we do not know about it yet.

The error(s) occur during or before GUNW processing, but downstream of SLC formation. This conclusion stems from the fact that the original processing of the Tibet scenes (v2_0_2) show no apparent misregistration from Google Earth basemap imagery. Therefore, it is not a problem with the SLCs or metadata provided by Copernicus. Instead, v2_0_4 GUNWs are misregistered, implying that something goes wrong during the ISCE or ARIA processing steps. The misregistration does not occur during extraction or stitching of the scenes in ARIA-tools, because the frames extracted directly from the NetCDF files show the same misregistration.

Note that not every v2_0_4 GUNW is affected. More than half of interferograms are not misregistered in either track.

To Reproduce This can be seen by downloading several example frames.

Examples include: S1-GUNW-A-R-041-tops-20141231_20141207-115952-38760N_36887N-PP-7560-v2_0_1.nc S1-GUNW-A-R-041-tops-20141231_20141207-115952-38760N_36887N-PP-6c2f-v2_0_4.nc

S1-GUNW-A-R-041-tops-20190912_20180905-120004-37600N_35557N-PP-6963-v2_0_2.nc S1-GUNW-A-R-041-tops-20170817_20160717-120015-390925N_37050N-PP-ed21-v2_0_4.nc

Of particular interest are the pair of adjacent GUNWs: S1-GUNW-A-R-041-tops-20141231_20141207-115927-37270N_35394N-PP-4485-v2_0_1.nc S1-GUNW-A-R-041-tops-20141231_20141207-115952-38760N_36887N-PP-7560-v2_0_1.nc versus S1-GUNW-A-R-041-tops-20141231_20141207-115927-37270N_35394N-PP-d9a6-v2_0_4.nc S1-GUNW-A-R-041-tops-20141231_20141207-115952-38760N_36887N-PP-6c2f-v2_0_4.nc

A complete list of date pairs with identified misregistrations is enumerated in the attached text document, "Track041_ExcludeRegistration.txt".

Expected behavior The expected behavior appears to be achieved by the original processing of the Tibet GUNWs for track 41. I.e., v20<1,2>

Track041_ExcludeRegistration.txt

[rzinke]

At the suggestion of @cmarshak, I checked the version number of the misregistered GUNWs. All were version 4. I checked several version 5 frames and all appeared to be properly georegistered and exhibited higher spatial coherence than the corresponding version 4 equivalents.

This appears to be related to the transition from HySDS to Hyp3.

[sssangha]

@rzinke , as a sanity check could you perhaps process a subset through the latest ARIA-tools PR.

One of the bug-fixes involves a patch where products were being grouped together where they shouldn't because they shared the same secondary scene, but had different reference scenes.

[rzinke]

I checked ~600 v2_0_5 frames and none of them showed the misregistration error. I'll reprocess with the latest AT overnight and see what it comes up with.

[dbekaert]

@rzinke: From your analysis i understand this is not an orbit issue as same day products only show the issue for version 4 and not version 1,2, and 5.

@cmarshak Could you confirm by replicating 1-2 failed version 4 products using our current version 5 code?

[cmarshak]

Excellent idea, David.

I just launched the jobs.

To follow up about the previous versions. These are products that were generated on the HySDS-HPC. You can tell because of the metadata fields in CMR.

[{'product_id': 'S1-GUNW-A-R-041-tops-20141231_20141207-115927-37270N_35394N-PP-d9a6-v2_0_4',
  'product_version': 'v2_0_4',
  'reference_scenes': ['S1A_IW_SLC__1SSV_20141231T115913_20141231T115940_003963_004C50_1CEA-local'],
  'secondary_scenes': ['S1A_IW_SLC__1SSV_20141207T115849_20141207T115916_003613_00445F_3EA7-local',
   'S1A_IW_SLC__1SSV_20141207T115914_20141207T115941_003613_00445F_AF5A-local',
   'S1A_IW_SLC__1SSV_20141207T115939_20141207T120007_003613_00445F_BA69-local']}]

The suffix -local is an artifact from porting HySDS to NASA Pleiades.

I will share the urls here early in the afternoon.

[cmarshak]

I used these two products:

• S1-GUNW-A-R-041-tops-20141231_20141207-115927-37270N_35394N-PP-d9a6-v2_0_4
• S1-GUNW-A-R-041-tops-20170817_20160717-120015-390925N_37050N-PP-ed21-v2_0_4

I downloaded the amplitude tiff from ASF and the water bodies definitely were not where they were supposed to be.

Here are the links for the regenerated products using the same inputs:

https://hyp3-a19-jpl-contentbucket-1wfnatpznlg8b.s3.us-west-2.amazonaws.com/2cbd18b7-7428-4b01-bb23-b0b1108ff9e0/S1-GUNW-A-R-041-tops-20141231_20141207-115926-00088E_00035N-PP-d9a6-v2_0_6.nc https://hyp3-a19-jpl-contentbucket-1wfnatpznlg8b.s3.us-west-2.amazonaws.com/ffea0ac1-93e2-432d-9848-7bea8d795f01/S1-GUNW-A-R-041-tops-20170817_20160717-120014-00088E_00037N-PP-d692-v2_0_6.nc

I didn't see the misregistration in the regenerated products when looking at the amplitude. @rzinke - please confirm and we can process using the Tibet AWS account.

[rzinke]

That worked perfectly. The newly reprocessed products are well-aligned with basemap imagery, and the filtered spatial coherence has noticeably improved.

Thank you all very much!