search

ASFHyP3 / hyp3-gamma

HyP3 plugin for generating SAR products with GAMMA
BSD 3-Clause "New" or "Revised" License
30 stars 7 forks source link

upgrade to gamma 20220630 #391

Closed asjohnston-asf closed 2 years ago

asjohnston-asf commented 2 years ago

Only significant change between the 202206030 linux installation guide and the one from 20210701 is the addition of python3-packaging. The 20220630 installation guides for ubuntu 20.04 and 22.04 are byte-for-byte identical.

I've run rtc for S1A_IW_SLC__1SSV_20150621T120220_20150621T120232_006471_008934_72D8 and insar for S1A_IW_SLC__1SSV_20150621T120220_20150621T120232_006471_008934_72D8+S1A_IW_SLC__1SSV_20150504T120217_20150504T120229_005771_00769E_EF9A on my local machine with all default options. They both run to completion and produce output that looks consistent to the naked eye in a GIS.

Will need to run a full test suite to exercise all the parameter options and code paths.

asjohnston-asf commented 2 years ago

I've been able to successfully run the six RTC jobs in https://github.com/ASFHyP3/hyp3-testing/blob/develop/hyp3_testing/templates/rtc_gamma_golden.json.j2 for both dem_type=copernicus and dem_type=legacy, as well as the eleven INSAR jobs proposed by @forrestfwilliams in https://github.com/ASFHyP3/hyp3-testing/pull/49 . They all completed successfully. I only looked at the browse image outputs, but they looked reasonable to the naked eye. That's enough to give me confidence that there are no fatally-breaking changes in this upgrade.

I think we'll be in a good position to merge once https://github.com/ASFHyP3/hyp3-testing/pull/49 is complete, so we can leverage the new testing implementation to compare the actual content of the finished products.

cirrusasf commented 2 years ago

I built the two versions of docker containers on my local machine. No. 1 is with gamma-20220630-ubuntu20.04, No.2 is with gamma-20220630-ubuntu22.04. I run rtc in Debug mode in No 1. It works. I run rtc without debug mode in No 2, it works. When I run rtc in debug mode in No 2, it fails. it complains os.path.sep * AttributeError: module 'posixpath' has no attribute 'sep'. The fail comes from hyp3lib.get_dem. I debug the get_dem.py , I found after from pyproj import Transformer, the os.path.sep attribute disappears. I can not find where the loaded module Transformer changes the os.path definition, but I can make the work around. I add the code

if not hasattr(os.path, 'sep'): os.path.sep ='/'

right before the beginning of the function definition in the get_dem.py. It works.

Andrew found that the pyproj issue was related to the failure of running hyp3-gamma-u22.04 in debug mode. The issue is addressed at https://github.com/pyproj4/pyproj/issues/1028. It says the problem will go away in the next release.

cirrusasf commented 2 years ago

I run the InSAR job successfully on my local machine with the newest docker container. I think we are ready to merge the branch to develop.

asjohnston-asf commented 2 years ago

:+1: @cirrusasf I've marked the PR ready for review so you can approve it, but we'll still hold off merging until @forrestfwilliams and @jhkennedy give us the go-ahead that https://github.com/ASFHyP3/hyp3-testing/pull/49 is merged and validated

cirrusasf commented 2 years ago

OK. We will waiting for them.

jhkennedy commented 2 years ago

Almost there -- golden tests have been merged and I've dispatched two runs:

Once those go green and we know testing is working we can merge this and kick off golden tests for this PR.