MintPy import error in ifgram_inversion_L1L2.py

[falkamelung]

With the test data I am getting this error. This was not an issue in MinoPy. I am using MintPy v1.4.1

<><><><><><><><><> step 8 - invert_network (MiaplPy) <><><><><><><><><><>
Generate /scratch2/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/run_files/run_08_miaplpy_invert_network
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/network_inversion.py", line 11, in <module>
    from miaplpy.dev import ifgram_inversion_L1L2
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/dev/ifgram_inversion_L1L2.py", line 24, in <module>
    from mintpy.utils import readfile, writefile, ptime, utils as ut, arg_group
ImportError: cannot import name 'arg_group' from 'mintpy.utils' (/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MintPy/mintpy/utils/__init__.py)

[yunjunz]

mintpy.utils.arg_group was renamed to mintpy.utils.arg_utils recently (https://github.com/insarlab/MintPy/pull/823).

[falkamelung]

Thank you @yunjunz ! Fixing this, the inversion works fine, but further down in the workflow I get this error. I would say the test data currently don't work. Or am I doing something stupid, @mirzaees ?

generate_temporal_coherence.py -d /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference
20220925:230401 * generate_temporal_coherence.py -d /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference
All updated (removed) attributes already exists (do not exists) and have the same value, skip update.
--------------------------------------------------
open  HDF5 file temporalCoherence.h5 in a mode
writing dataset /temporalCoherence         block: [0, 530, 0, 1685]
close HDF5 file temporalCoherence.h5.
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/generate_temporal_coherence.py", line 173, in <module>
    main()
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/generate_temporal_coherence.py", line 164, in main
    get_phase_linking_coherence_mask(metadata, inps.work_dir)
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/generate_temporal_coherence.py", line 32, in get_phase_linking_coherence_mask
    tcoh_min = float(template['miaplpy.timeseries.minTempCoh'])
KeyError: 'miaplpy.timeseries.minTempCoh'

[mirzaees]

Thanks @yunjunz! @falkamelung There were a few changes in MintPy which miaplpy needed to adapt. I have now committed a new PR, please check if the problems are resolved.

[falkamelung]

Thank you @mirzaees . The testdata now work for me!

But there is an issue with the run_files. They are now at PichinchaSenDT142/miaplpy/network_single_reference/run_files/ which does not look right - and did not work for me. For MiNoPy they were at PichinchaSenDT142/miaplpy/run_files. Is this a mistake?

[mirzaees]

@falkamelung That is intentionally If you remember, we did it to be able to run different networks at the same time without the need to delete or relocate run files. What do you mean it did not work for you? There is no problem when I run it

[falkamelung]

Hi @mirzaees I see. After changing this in run_workflow.bash it goes until the load_ifgram step. So to run different networks I need to modify run_workflow.bash to optionally accept other run_files directories on the command line?

The error in the load_ifgram step occurs for the command in the run_file. It does not produce /inputs/geometryRadar.h5. Running miaplpyApp.py ./PichinchaSenDT142.txt --dostep load_ifgram works fine. Maybe this is also related to the MintPy update?

cat run_06_miaplpy_load_ifgram 
load_ifgram.py --template /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/miaplpyApp.cfg /scratch/05861/tg851601/PichinchaSenDT142/PichinchaSenDT142.txt --project PichinchaSenDT142 --output /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/ifgramStack.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryRadar.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryGeo.h5

load_ifgram.py --template /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/miaplpyApp.cfg /scratch/05861/tg851601/PichinchaSenDT142/PichinchaSenDT142.txt --project PichinchaSenDT142 --output /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/ifgramStack.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryRadar.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryGeo.h5

20220930:070036 * load_ifgram.py --template /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/miaplpyApp.cfg /scratch/05861/tg851601/PichinchaSenDT142/PichinchaSenDT142.txt --project PichinchaSenDT142 --output /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/ifgramStack.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryRadar.h5 /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryGeo.h5
SAR platform/sensor : Sen
check auto path setting for Univ of Miami users for processor: isce
--------------------------------------------------
prepare metadata files for isce products
prep_isce.py -m /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inputs/reference/IW3.xml -g ../merged/geom_reference -b /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inputs/baselines  -f ./inverted/interferograms_single_reference/*/*fine*.unw 
['/scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inputs/reference/data.rsc'] exists and is newer than ['/scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inputs/reference/IW3.xml'] --> skip.
preparing RSC file for geometry files
read perp baseline time-series from /scratch/05861/tg851601/PichinchaSenDT142/miaplpy/inputs/baselines
preparing RSC file for: ./inverted/interferograms_single_reference/*/*fine*.unw
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MintPy/mintpy/cli/prep_isce.py", line 116, in <module>
    main(sys.argv[1:])
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MintPy/mintpy/cli/prep_isce.py", line 111, in main
    prep_isce(inps)
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MintPy/mintpy/prep_isce.py", line 214, in prep_isce
    prepare_stack(
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MintPy/mintpy/prep_isce.py", line 137, in prepare_stack
    raise FileNotFoundError(f'NO file found with path pattern: {obs_file}')
FileNotFoundError: NO file found with path pattern: ./inverted/interferograms_single_reference/*/*fine*.unw
WARNING: mintpy.subset.lalo is not supported if 1) no lookup file AND    2) radar/unkonwn coded dataset
    ignore it and continue.
--------------------------------------------------
searching interferogram pairs info
input data files:
WARNING: No data files found for the required dataset: ['unwrapPhase']! Skip loading for interferogram stack.
--------------------------------------------------
updateMode : True
compression: None
Traceback (most recent call last):
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/load_ifgram.py", line 472, in <module>
    main()
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/sources/MiaplPy/miaplpy/load_ifgram.py", line 405, in main
    shutil.copyfile(geometry_file, os.path.join(work_dir, 'inputs/{}'.format(geometry_file_2)))
  File "/work2/05861/tg851601/stampede2/test3_frontera/rsmas_insar/3rdparty/miniconda3/lib/python3.9/shutil.py", line 266, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/scratch/05861/tg851601/PichinchaSenDT142/miaplpy/network_single_reference/inputs/geometryRadar.h5'

[mirzaees]

Hi @falkamelung, The directories given in template file are not full path, so it shows error when you are in the wrong directory. I committed a quick fix.

[falkamelung]

Hi @mirzaees Thank you! This works fine now! So I will do the run_workflow.bash as suggested above and have minsarApp.bash check for which network is selected.

[falkamelung]

closing as resolved