search

GeoscienceAustralia / ga_sar_workflow

InSAR processing workflow used by Geoscience Australia
Apache License 2.0
3 stars 0 forks source link

Add support for handling multiple GAMMA versions in a single code base #360

Closed mwheeler closed 2 years ago

mwheeler commented 2 years ago

This issue is to solve the problem that GAMMA's "api" is not stable (or even consistent) across versions, thus different versions may need programs called differently (or different programs entirely to be called) to achieve the same thing.

After already prototyping the upgrade to 20210701 manually and looking at how GAMMA typically seems to change across versions, the changes don't seem too major (but they're absolutely breaking changes).

I think the best approach to this is to promote the unit testing GAMMA proxy to the wrapper we use in the whole code base, which provides the following benefits:

  1. It has a solid code structure (the py_gamma_ga.py wrapper is fully dynamic / has no structure for code completion or linting)
  2. It's also well typed (even better linting)
  3. We can upgrade it to bring documentation directly into the coding environment as docstrings
  4. It supports proxies already (so under the hood it can still call into py_gamma_ga.py retaining identical behaviour)
  5. It's a standard python class so we can inherit it & extend/override functions for each specific version of GAMMA, solving our need to keep a stable API across GAMMA versions
mwheeler commented 2 years ago

Historical context for GAMMA version upgrade: #249

mwheeler commented 2 years ago

As part of this work, I've observed newer versions of GAMMA fix some artefacts that were present in the older processing - I've attached a before/after example over one of my test scenes (Melbourne) - however we observed these sorts of burst boundary artefacts reasonably often in the big chunk of processing we did for 2016+

In the examples below (backscatter), notice around the bottom left of the first image (processed w/ 20191203) that along the burst boundaries there are discontinuities in brightness/intensity of the backscatter signal. These magically disappear in the newer GAMMA versions / seem to be fixed.

It's not "as" obvious, but there's similar discontinuities along the burst boundaries about 2/3rds of the way to the right of the image where the 2nd and 3rd subswaths join, which are also fixed in the newer version)

GAMMA version 20191203: 20210202_VV

GAMMA version 20210701: 20210202_VV

The data for these tests is available for analysis at: 20191203: /g/data/dz56/mitch-testing/S1/GAMMA/out_dir/T118D_F38S_S1A_20191203 20210701: /g/data/dz56/mitch-testing/S1/GAMMA/out_dir/T118D_F38S_S1A_proxy_20210701 20211208: /g/data/dz56/mitch-testing/S1/GAMMA/out_dir/T118D_F38S_S1A_proxy_20211208