Poor focusing using the ROI_PAC sensor type in stripmapApp

[geniusinaction]

This is an edge case for sure, but I am trying to process legacy JERS data from raw using stripmapApp. Since there is no example input file format (that I could find in the current version, anyway) for JERS, I am trying to use the ROI_PAC input format. I did once successfully process this pair with ROI_PAC, so this was pretty easy to set up, with the raw files, rsc files and all that.

The results, however, were disappointing. Neither image is well focused at the SLC stage, and everything beyond that is (as you might expect) fuzzy garbage. I am left wondering if there is something in the metadata parsing that did not make it from ROI_PAC into ISCE? I note that both the raw files have significant Doppler shifts (-1.2 PRF in the ROI_PAC nomenclature) which does make me wonder if that might be the issue? Anyone have any suggestions?

[piyushrpt]

Check units of _dopplerVsRangePixel vs that of another sensor like TSX / CSK / ALOS. It is possible that they are normalized by PRF in ROI_PAC; and might need to be converted to Hz before using with any of the workflows.

https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/components/isceobj/Sensor/ROI_PAC.py#L223

[geniusinaction]

Intriguing! I assume the values of doppler_vs_pixel represent a static coefficient, linear and quadratic terms? I am not sure if the numbers agree exactly.

This is what I get:

My JERS image (selected entries from isce.log):

reference.frame.doppler_vs_pixel = [-284.0383895212319, -0.08412333288059079, 2.1196022751847877e-06]
reference.instrument.range_pixel_size = 8.76586134502924
reference.sensor.DOPPLER_RANGE0 = -1.2416044171086
reference.sensor.DOPPLER_RANGE1 = -4.42229572736922e-05
reference.sensor.DOPPLER_RANGE2 = 0
reference.sensor.PRF = 1555.2000000

-1.24 1555 = -1920 (with the integer ambiguity included, compare with -284) -0.24 1555 = -373 (without the integer ambiguity, compare with -284) -4.42e-05 * 1555 / 8.76 = -0.00786 (compare with -0.0841)

Might this mean that the integer ambiguity is not being passed? The difference in gradients is still an order of magnitude, though.

An ERS scene, that I processed with ISCE (using the ERS.py - i.e. raw - parser) and ROI_PAC:

reference.frame.doppler_vs_pixel = [366.8354526906382, -0.007878482870533469, 1.9696267475015959e-07]
reference.instrument.prf = 1679.8784549964564
reference.instrument.range_pixel_size = 7.904890281159604

and from the ROI_PAC processing of the same data:

DOPPLER_RANGE0                           0.2232
DOPPLER_RANGE1                           -5.0521e-07
DOPPLER_RANGE2                           -5.0384e-10

0.2232 1680 = 374.9 (compared with 366.8) -5.052e-07 1680 / 7.905 = -0.000107 (compared with -0.00788)

These numbers are closer to the ISCE equivalents?

[piyushrpt]

Yeah . it looks like the ambiguity is not being passed through correctly. ROI_PAC itself had a couple of conventions and it was not always clear. The first one was that coefficients were a function of pixel number w.r.t starting range of the scene and the second one was coefficients as a function of pixel number w.r.t zero starting range. Might have to dig into JERS parser to see what it was spitting out. ISCE sensor supports the first one. But my first guess is that DOPPLER_RANGE{} is not getting translated correctly to doppler_vs_pixel

[geniusinaction]

Digging a little deeper, it seems that the default for ROI_PAC-format data is to run DopIQ and not to use the information from the file headers.

https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/components/isceobj/StripmapProc/Factories.py#L79-L82

In an extremely hacky way, I temporarily added ROI_PAC to the default list, to see what happens if I forced it to use DefaultDopp.py instead of DopIQ.py. I found that:

1. The reported Doppler vs pixel coefficients, are a very large first number (= the Doppler shift in Hz * the PRF), and zeros otherwise - it seems that the other Doppler coefficients are not used at all.
2. If I then hacked the ROI_PAC.py parser, to divide the first coefficient by the PRF, I can produce adequately focused reference and secondary SLCs, with constant Doppler shifts of ~-1900 Hz for both. However, these SLCs didn't coregister properly (there is a biggish range shift between them, perhaps because the Doppler gradients are not accounted for?) and the interferogram is still junk.

The improved SLCs:

So I guess there a couple of possible approaches? Pass the integer ambiguity to DopIQ somehow, or hack DefaultDopp and friends to use the values of the Doppler coefficients from the rsc files properly. Assuming that the latter is easier, how would I go about doing that?

[piyushrpt]

I thought with the sensors, you could just do this - <property name="doppler method">useDEFAULT</property>

I could be wrong - its been a long time since I used this. I thought that the idea was to fall back to that bit of code if nothing is specified by the user. ROI_PAC should definitely fall in the default category.

[piyushrpt]

Also I see ROI_PAC sensor setting dopplerVsRangePixel vs _dopplerVsPixel used by default doppler . This reader may not have ever been updated to work with stripmapApp. That might explain the metadata not being propagated correctly.

[geniusinaction]

Thanks for the pointer, it seems the correct properties are:

<property name="reference doppler method">useDEFAULT</property>
<property name="secondary doppler method">useDEFAULT</property>\

If I use this without any additional code hacks, I get these:

reference.frame.doppler_vs_pixel = [-3003002.848290641, 0.0, 0.0, 0.0]
secondary.frame.doppler_vs_pixel = [-2882779.5730339526, 0.0, 0.0, 0.0]

Which are both a factor of the PRF too large, and the other coefficients are not passed somehow. Manually hacking the ROI_PAC.py to divide dopp[0] by the PRF is how I managed to get the focused SLCs in my previous post.

I guess the question is how to get ISCE to use the quadratic coefficients from ROI_PAC.py in the rest of the processing workflow?

[piyushrpt]

What you need is this https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/contrib/stack/stripmapStack/unpackFrame_ROIPAC_raw.py#L68

inserted here in place of https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/components/isceobj/Sensor/ROI_PAC.py#L223

Scale what you read out of the rsc by the PRF as shown in the unpack script.

If you get the doppler wrong, topo will be way wrong - you can see z.rdr not line up with the reference SLC.

[geniusinaction]

OK, have changed instances of dopplerVsRangePixel to dopplerVsPixel in ROI_PAC.py. Under _populateExtras I now have

mdict = self.constants
prf=float(mdict['PRF'])
self.frame._dopplerVsPixel = [float(mdict['DOPPLER_RANGE0'])*prf,
                              float(mdict['DOPPLER_RANGE1'])*prf,
                              float(mdict['DOPPLER_RANGE2'])*prf ]

and later on, under extractDoppler, I have

self._populateExtras()
quadratic = {}
dopp = self.frame._dopplerVsPixel
mid = 0.5*(int(self.constants['WIDTH']) - int(self.constants['XMIN']))
fd_mid = dopp[0] + (dopp[1] + dopp[2]*mid)*mid

quadratic['a'] = dopp[0]
quadratic['b'] = dopp[1]
quadratic['c'] = dopp[2]

return quadratic

And now it crashes this part of make_raw.py, which is expecting values normalized by PRF, it seems!

https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/applications/make_raw.py#L204-L221

I guess I can add another if statement here to check the size of the values?

[piyushrpt]

a, b, c is from insarApp days. When setting those, just divide by PRF again. It is just calculating a bunch of values like squint etc that are not needed

[geniusinaction]

This is my hacky fix to lines 204 and 205 above. Seems to be working (SLCs focusing, etc).

if 'a' in self.doppler.quadratic:
    # hacky way to check whether normalized by prf or not
    if abs(self.doppler.quadratic['a']) > 3:
       fd = self.doppler.quadratic['a']    
    else:
       fd = self.doppler.quadratic['a']*prf

[geniusinaction]

I made that change before I saw the comment about dividing a, b and c by the PRF. I'll try that in a little bit.

[geniusinaction]

OK, so with those changes to ROI_PAC.py, the two SLCs focus sensibly (which they didn't originally), which is nice. Updated script is here, for posterity:

ROI_PAC.py.txt

But there is still a coregistration issue. The two SLCs seem to be misregistered in azimuth, so there are no fringes, even though the amplitudes are a little sharper than the first time I tried...

Any advice on this part of the problem? I thought topo registration was supposed to prevent this from happening, but it seems not...

[piyushrpt]

Misregistration in azimuth is understandable - this is because of clocks not being synced very well - radar clock vs GPS clock; in earlier missions. Range should line up really nicely since ranging is always more accurate. For older sensors, you will need to use larger search windows to estimate the azimuth correction.

Here is where the values need to be changed: https://github.com/isce-framework/isce2/blob/60a021de430ce2b410f9ede03064b1e8588e1db1/components/isceobj/StripmapProc/runRefineSecondaryTiming.py#L41

Its currenly looking at a window of +/-40 lines and +/-40 pixels. You might only need to change the lines search range. and you can just rerun from that step.

[piyushrpt]

And the changes are best submitted as a PR to the github repo

[geniusinaction]

With an 80 search pixel window in azimuth, I get... fringes!

Well, I never! (Time to play the game, orbital ramp or ionosphere?)

Thanks for all your help, Piyush! I'll see what I can do, w.r.t. pull requests...

[piyushrpt]

There is definitely a huge orbit component in JERS / Radarsat-1. Ionosphere as well likely in JERS

[geniusinaction]

Generated pull request #641 with fixes