search

insarlab / MintPy

Miami InSAR time-series software in Python
https://mintpy.readthedocs.io
Other
555 stars 244 forks source link

error in prep_gmtsar : not reading *prm files #1194

Closed soroushyasini closed 1 month ago

soroushyasini commented 1 month ago

hi, i used GMTSAR 6.4 as my processor for 21 Envisat descending images which generated 36 interfrograms. i also write a .csh code to make baseline.txt for each interferogram folder by two existing *.PRM files. i do believe my interferogram stack is ok, but when running mintpy main code smallbaselineApp.py i've the following error :

UN-at-2024-05-12 16:55:16.768619-- Current directory: /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270 Run routine processing with smallbaselineApp.py on steps: ['load_data', 'modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_ionosphere', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5'] Remaining steps: ['modify_network', 'reference_point', 'quick_overview', 'correct_unwrap_error', 'invert_network', 'correct_LOD', 'correct_SET', 'correct_ionosphere', 'correct_troposphere', 'deramp', 'correct_topography', 'residual_RMS', 'reference_date', 'velocity', 'geocode', 'google_earth', 'hdfeos5']

Project name: glb_GMT_dsc Go to work directory: /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270 copy default template file /home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/defaults/smallbaselineApp.cfg to work directory read custom template file: /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/glb_GMT_dsc.txt update default template based on input custom template mintpy.load.processor: auto --> gmtsar mintpy.load.unwFile: auto --> /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms//unwrap_ll.grd mintpy.load.corFile: auto --> /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms//corr_ll.grd mintpy.load.demFile: auto --> /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/topo/topo_ll.grd copy glb_GMT_dsc.txt to inputs directory for backup. copy smallbaselineApp.cfg to inputs directory for backup. copy glb_GMT_dsc.txt to pic directory for backup. copy smallbaselineApp.cfg to pic directory for backup. read default template file: /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/smallbaselineApp.cfg step - load_data load_data.py --template /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/smallbaselineApp.cfg /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/glb_GMT_dsc.txt --project glb_GMT_dsc processor : gmtsar SAR platform/sensor : unknown from project name "glb_GMT_dsc" prepare metadata files for gmtsar products prep_gmtsar.py /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/glb_GMT_dsc.txt /home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/load_data.py:777: UserWarning: prep_gmtsar.py failed. Assuming its result exists and continue... warnings.warn('prep_gmtsar.py failed. Assuming its result exists and continue...') updateMode : True compression: None multilook x/ystep: 1/1 multilook method : nearest /home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/osgeo/gdal.py:312: FutureWarning: Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default. warnings.warn( searching geometry files info input data files: height : /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/topo/topo_ll.grd

create HDF5 file /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/2003095_2003270/inputs/geometryGeo.h5 with w mode create dataset /height of <class 'numpy.float32'> in size of (2300, 2020) with compression = lzf geocoded input, use constant value from metadata INCIDENCE_ANGLE prepare slantRangeDistance ... Traceback (most recent call last): File "/home/gmt6/miniconda3/envs/mintpy/bin/smallbaselineApp.py", line 10, in sys.exit(main()) ^^^^^^ File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/cli/smallbaselineApp.py", line 209, in main run_smallbaselineApp(inps) File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 1155, in run_smallbaselineApp app.run(steps=inps.runSteps) File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 908, in run self.run_load_data(sname) File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/smallbaselineApp.py", line 176, in run_load_data mintpy.cli.load_data.main(iargs) File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/cli/load_data.py", line 123, in main load_data(inps) File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/load_data.py", line 840, in load_data geom_geo_obj.write2hdf5( File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/objects/stackDict.py", line 794, in write2hdf5 f.attrs[key] = value


  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/h5py/_hl/attrs.py", line 104, in __setitem__
    self.create(name, data=value)
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/h5py/_hl/attrs.py", line 182, in create
    htype = h5t.py_create(original_dtype, logical=True)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "h5py/h5t.pyx", line 1663, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1687, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1747, in h5py.h5t.py_create
TypeError: Object dtype dtype('O') has no native HDF5 equivalent

### I also ran prep_gmtsar.py for getting more info and here is the result : 

prep_gmtsar.py glb_GMT_dsc.txt 
Traceback (most recent call last):
  File "/home/gmt6/miniconda3/envs/mintpy/bin/prep_gmtsar.py", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/cli/prep_gmtsar.py", line 57, in main
    prep_gmtsar(inps)
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/prep_gmtsar.py", line 265, in prep_gmtsar
    meta = extract_gmtsar_metadata(
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/prep_gmtsar.py", line 119, in extract_gmtsar_metadata
    prm_file = get_prm_files(ifg_dir)[0]
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/gmt6/miniconda3/envs/mintpy/lib/python3.11/site-packages/mintpy/prep_gmtsar.py", line 27, in get_prm_files
    raise FileExistsError('NO two *.prm files found!')
FileExistsError: NO two *.prm files found!

############################################

### here is my config file (load data part) template : 

1. load_data
-##---------add attributes manually
-## MintPy requires attributes listed at: https://mintpy.readthedocs.io/en/latest/api/attributes/
-## Missing attributes can be added below manually (uncomment #), e.g.
HEADING         = -168.0   
ORBIT_DIRECTION = DESCENDING
#PLATFORM = CSK
-# ...
-## a. autoPath - automatic path pattern defined in mintpy.defaults.auto_path.AUTO_PATH_*
-## b. load_data.py -H to check more details and example inputs.
-# c. compression to save disk usage for ifgramStack.h5 file:
-## no   - save   0% disk usage, fast [default]
-## lzf  - save ~57% disk usage, relative slow
-## gzip - save ~62% disk usage, very slow [not recommend]
mintpy.load.processor       = gmtsar  #[isce, aria, hyp3, gmtsar, snap, gamma, roipac, nisar], auto for isce
mintpy.load.autoPath        = auto  #[yes / no], auto for no, use pre-defined auto path
mintpy.load.updateMode      = auto  #[yes / no], auto for yes, skip re-loading if HDF5 files are complete
mintpy.load.compression     = auto  #[gzip / lzf / no], auto for no.

-##---------interferogram stack:
mintpy.load.unwFile         = /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/*/unwrap_ll.grd
mintpy.load.corFile         = /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/interferograms/*/corr_ll.grd
mintpy.load.connCompFile    = auto  #[path pattern of connected components    files], optional but recommended
mintpy.load.intFile         = auto #[path pattern of wrapped interferogram   files], optional
mintpy.load.magFile         = auto  #[path pattern of interferogram magnitude files], optional
-##---------ionosphere stack (optional):
-##---------offset stack (optional):
mintpy.load.azOffFile       = auto  #[path pattern of azimuth offset file]
mintpy.load.rgOffFile       = auto  #[path pattern of range   offset file]
mintpy.load.azOffStdFile    = auto  #[path pattern of azimuth offset variance file], optional but recommended
mintpy.load.rgOffStdFile    = auto  #[path pattern of range   offset variance file], optional but recommended
mintpy.load.offSnrFile      = auto  #[path pattern of offset signal-to-noise ratio file], optional
-##---------geometry:
mintpy.load.demFile         = /media/gmt6/HDD/SAR_procs/golbahar/glb_GMT_dsc/topo/topo_ll.grd
mintpy.load.lookupYFile     = auto  #[path of latitude /row   /y coordinate file], not required for geocoded data
mintpy.load.lookupXFile     = auto  #[path of longitude/column/x coordinate file], not required for geocoded data
mintpy.load.incAngleFile    = auto  #[path of incidence angle file], optional but recommended
mintpy.load.azAngleFile     = auto  #[path of azimuth   angle file], optional
mintpy.load.shadowMaskFile  = auto  #[path of shadow mask file], optional but recommended
mintpy.load.waterMaskFile   = auto  #[path of water  mask file], optional but recommended
mintpy.load.bperpFile       = auto  #[path pattern of 2D perpendicular baseline file], optional
-##---------subset (optional):
-## if both yx and lalo are specified, use lalo option unless a) no lookup file AND b) dataset is in radar coord
mintpy.subset.yx            = auto    #[y0:y1,x0:x1 / no], auto for no
mintpy.subset.lalo          = auto    #[S:N,W:E / no], auto for no
mintpy.multilook.method     = auto    #[nearest, mean, median], auto for nearest - lines/rows skipping approach
mintpy.multilook.ystep      = auto    #[int >= 1], auto for 1 - no multilooking
mintpy.multilook.xstep      = auto    #[int >= 1], auto for 1 - no multilooking`

### and finally, here is my main file structure :

├── baseline.csh
├── baseline.ps
├── baseline_table.dat
├── glb_GMT_dsc.txt
├── gmt.history
├── inputs
├── interferograms
├── intf.in
├── pic
├── raw
├── SLC
├── smallbaselineApp.cfg
└── topo

and here is interferogram folder :
├── 2003270_2003340
├── 2003270_2004045
├── 2003340_2004045
├── 2003340_2004115
├── 2004045_2004115
├── 2004045_2004220
....

### and here is inside of one of them :

.
├── amp1.grd
├── amp2.grd
├── amp.grd
├── ASA_IMS_1PNESA20030405_061630_000000182015_00163_05728_0000.LED -> ../../SLC/ASA_IMS_1PNESA20030405_061630_000000182015_00163_05728_0000.LED
├── ASA_IMS_1PNESA20030405_061630_000000182015_00163_05728_0000.PRM
├── ASA_IMS_1PNESA20030405_061630_000000182015_00163_05728_0000.SLC -> ../../SLC/ASA_IMS_1PNESA20030405_061630_000000182015_00163_05728_0000.SLC
├── ASA_IMS_1PNESA20030927_061640_000000182020_00163_08233_0000.LED -> ../../SLC/ASA_IMS_1PNESA20030927_061640_000000182020_00163_08233_0000.LED
├── ASA_IMS_1PNESA20030927_061640_000000182020_00163_08233_0000.PRM
├── ASA_IMS_1PNESA20030927_061640_000000182020_00163_08233_0000.SLC -> ../../SLC/ASA_IMS_1PNESA20030927_061640_000000182020_00163_08233_0000.SLC
├── baseline.txt
├── conncomp.grd
├── corr.cpt
├── corr.grd
├── corr_ll.grd
├── corr_ll.kml
├── corr_ll.png
├── corr.pdf
├── display_amp.cpt
├── display_amp.grd
├── display_amp_ll.grd
├── display_amp_ll.kml
├── display_amp_ll.png
├── display_amp.pdf
├── filtcorr.grd
├── gauss_200
├── glb_GMT_dsc.txt
├── gmt.conf
├── gmt.history
├── ijdec
├── imagfilt.grd
├── landmask_ra.grd -> ../../topo/landmask_ra.grd
├── landmask_ra_patch.grd
├── los.cpt
├── los_grad.grd
├── los.grd
├── los_ll.grd
├── los_ll.kml
├── los_ll.png
├── los.pdf
├── mask2.grd
├── mask2_patch.grd
├── mask3.grd
├── mask.grd
├── phase.cpt
├── phasefilt.grd
├── phasefilt_ll.grd
├── phasefilt_mask.grd
├── phasefilt_mask_ll.grd
├── phasefilt_mask_ll.kml
├── phasefilt_mask_ll.png
├── phasefilt_mask.pdf
├── phasefilt.pdf
├── phase.grd
├── phase_mask.grd
├── phase_mask_ll.grd
├── phase_mask_ll.kml
├── phase_mask_ll.png
├── phase_mask.pdf
├── phase_patch.grd -> phasefilt.grd
├── phase.pdf
├── raln.grd
├── ralt.grd
├── realfilt.grd
├── topo_ra.grd -> ../../topo/topo_ra.grd
├── trans.dat -> ../../topo/trans.dat
├── unwrap.cpt
├── unwrap.grd
├── unwrap_ll.grd
├── unwrap_mask.grd
├── unwrap_mask_ll.grd
├── unwrap_mask_ll.kml
├── unwrap_mask_ll.png
├── unwrap_mask.pdf
└── unwrap.pdf 

as you can see two *.PRM files exist in each of interferogram folder ( in fact they are link files to SLC folder orginal *PRM) files.

any opinion ?
welcome[bot] commented 1 month ago

👋 Thanks for opening your first issue here! Please filled out the template with as much details as possible. We appreciate that you took the time to contribute! Make sure you read our contributing guidelines.