search

GeoscienceAustralia / PyRate

A Python tool for estimating velocity and time-series from Interferometric Synthetic Aperture Radar (InSAR) data.
https://geoscienceaustralia.github.io/PyRate/
Apache License 2.0
201 stars 71 forks source link

Sb/restart from master #259

Closed basaks closed 4 years ago

basaks commented 4 years ago
  1. sprint_11 + eliminate_linrate + run on gadi + update_troubleshooting + 'file-handling' PR functionalities from Nov 25 + Jan 14 merged on a fork of master. Please check if I have missed anything and merge into develop. By sprint_11 PR tests were already broken. I have fixed tests.
  2. I have introduced travis CI using python3.6 and python 3.7 using gdal 3.0.2 and 3.0.4. One of these is our NCI gdal/python combination.
  3. I have published a dockerfile which we could use in future in travis/local development.

Next step wll be to merge load utils config and bug-single-mask-coherence PRs into develop.

codecov-io commented 4 years ago

Codecov Report

Merging #259 into develop will not change coverage. The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop     #259   +/-   ##
========================================
  Coverage    90.46%   90.46%           
========================================
  Files           26       26           
  Lines         3389     3389           
  Branches       574      574           
========================================
  Hits          3066     3066           
  Misses         210      210           
  Partials       113      113

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 3c35b46...3c35b46. Read the comment docs.

sheecegardezi commented 4 years ago

I can confirm that functionality from sprint 11 have been ported in this PR.

mcgarth commented 4 years ago

Update: this issue confirmed fixed by 0e664f6

done some prepifg user tests on commit https://github.com/GeoscienceAustralia/PyRate/pull/259/commits/47e6fd22ad3927a9c9a29309578283851f696108

test location: /g/data/dg9/mcg547/VICTORIA_S1_T045D_F21 config file: input_parameters.conf_new

param highlights: cohmask: 1 cohthresh: 0.1 ifgcropopt: 3 ifglksx/y: 20 parallel: 0 mpi: no

16:40:26 shared 1294 45456 0/0 Running prepifg serially
20151202-20151214
/g/data/dz56/INSAR_ARD/VV/INSAR_ANALYSIS/VICTORIA/S1/GAMMA/T45D_F21/INT/20151202-20151214/20151202-20151214_VV_4rlks_eqa_unw.tif
Traceback (most recent call last):
  File "/home/547/mcg547/PyRateVenv/bin/pyrate", line 11, in <module>
    load_entry_point('Py-Rate==0.4.0', 'console_scripts', 'pyrate')()
  File "/home/547/mcg547/repo/PyRate/pyrate/main.py", line 119, in main
    prepifg_handler(args.config_file)
  File "/home/547/mcg547/repo/PyRate/pyrate/main.py", line 47, in prepifg_handler
    prepifg.main(config.__dict__)
  File "/home/547/mcg547/repo/PyRate/pyrate/prepifg.py", line 57, in main
    do_prepifg(gtiff_paths, params)
  File "/home/547/mcg547/repo/PyRate/pyrate/prepifg.py", line 88, in do_prepifg
    _prepifg_multiprocessing(gtiff_path, xlooks, ylooks, exts, thresh, crop, params)
  File "/home/547/mcg547/repo/PyRate/pyrate/prepifg.py", line 105, in _prepifg_multiprocessing
    coherence_path = cf.coherence_paths_for(path, params, tif=True)
  File "/home/547/mcg547/repo/PyRate/pyrate/core/config.py", line 506, in coherence_paths_for
    coh_file_paths = [f.converted_path for f in params[COHERENCE_FILE_PATHS] if epoch in f.unwrapped_path]
  File "/home/547/mcg547/repo/PyRate/pyrate/core/config.py", line 506, in <listcomp>
    coh_file_paths = [f.converted_path for f in params[COHERENCE_FILE_PATHS] if epoch in f.unwrapped_path]
TypeError: argument of type 'NoneType' is not iterable

The same job with coherence masking turned off (cohmask = 0) worked fine.

sheecegardezi commented 4 years ago

Comparing https://github.com/GeoscienceAustralia/PyRate/tree/test with https://github.com/GeoscienceAustralia/PyRate/tree/sb/restart-from-master as of today (511ea07f3d96b47ba42fbd9cede9bf0f5f1022b6), following is list of things that are missing: