Closed alejobeap closed 3 years ago
welcome[bot]
commented
4 years ago 👋 Thanks for opening your first issue here! Please make sure you filled out the template with as much detail as possible. We appreciate that you took the time to contribute! Make sure you read our contributing guidelines
yunjunz
commented
4 years ago @alejobeap is this interferogram stack in full resolution, without any multilooking applied in stripmapStack processing?
number of independent looks L=0 looks suspicious to me, but I am not sure if this is the cause. If it is, it should be solved by the commit below.
yunjunz
commented
4 years ago any update on this @alejobeap ?
alejobeap
commented
4 years ago Dear I'm trying again Firts I did (example)
unpackFrame_TSX.py -i /mnt/datos_sentinel/Datos_TerraSarX/SANGAY/Descendente/TSX_20200708T105955.647_LatinAmerica_C434_O142_D_R_SM011_SSC/dims_op_oc_dfd2_644491745_1/ -o /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/SLC/
For each image
Then I run stackStripMap.py -s /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/SLC/ -d /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/DEM/demLat_S03_S01_Lon_W079_W077.dem.wgs84 --bbox '-2.04 -1.97 -78.39 -78.28' -w ./ -W interferogram -t 120 -b 400 -m 20200103 --nofocus --zero
Is correct??
I am still running at ISCE,
Thanks
Pedro
yunjunz
commented
4 years ago Now I can see that you were running stack processing in full resolution, which should be the cause of the error before, this is now fixed.
Your calling of stripmapStack looks good to me. Unless the full resolution is desired, I would recommend applying some multilooking (stackStripMap.py ---azimuth_looks --range_looks) to reduce the computing/disk load and increase the signal-to-noise ratio.
alejobeap
commented
4 years ago Dear I did according your instructions but i have the same error: number of independent looks L=0
******************** step - invert_network ********************
ifgram_inversion.py /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/ifgramStack.h5 -t /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/smallbaselineApp.cfg --update
use dataset "unwrapPhase" by default
--------------------------------------------------
update mode: ON
1) NOT ALL output files found: ['timeseries.h5', 'temporalCoherence.h5', 'numInvIfgram.h5'].
run or skip: run.
reference pixel in y/x: (1409, 2714) from dataset: unwrapPhase
-------------------------------------------------------------------------------
least-squares solution with L2 min-norm on: deformation velocity
minimum redundancy: 1.0
weight function: var
mask: no
-------------------------------------------------------------------------------
number of interferograms: 11
number of acquisitions : 12
number of lines : 3295
number of columns : 4268
--------------------------------------------------
create HDF5 file timeseries.h5 with w mode
create dataset: date of |S8 in size of (12,)
create dataset: bperp of <class 'numpy.float32'> in size of (12,)
create dataset: timeseries of <class 'numpy.float32'> in size of (12, 3295, 4268)
close HDF5 file timeseries.h5
--------------------------------------------------
open HDF5 file timeseries.h5 in a mode
writing dataset /date block: [0, 12]
close HDF5 file timeseries.h5.
--------------------------------------------------
open HDF5 file timeseries.h5 in a mode
writing dataset /bperp block: [0, 12]
close HDF5 file timeseries.h5.
--------------------------------------------------
create HDF5 file temporalCoherence.h5 with w mode
create dataset: temporalCoherence of <class 'numpy.float32'> in size of (3295, 4268)
close HDF5 file temporalCoherence.h5
--------------------------------------------------
create HDF5 file numInvIfgram.h5 with w mode
create dataset: mask of <class 'numpy.float32'> in size of (3295, 4268)
close HDF5 file numInvIfgram.h5
maximum memory size: 2.0E+00 GB
split 3295 lines into 5 patches for processing
with each patch up to 700 lines
------- processing patch 1 out of 5 --------------
box width: 4268
box length: 700
reading coherence in (0, 0, 4268, 700) * 11 ...
convert coherence to weight in chunks of 100000 pixels: 30 chunks in total ...
convert coherence to weight using inverse of phase variance
with phase PDF for distributed scatterers from Tough et al. (1995)
number of independent looks L=0
/home/insar/tools/MintPy/mintpy/simulation/decorrelation.py:105: RuntimeWarning: invalid value encountered in less_equal
flag = var <= 0
/home/insar/tools/MintPy/mintpy/simulation/decorrelation.py:107: RuntimeWarning: All-NaN slice encountered
var[flag] = np.nanmin(var[~flag])
chunk 1 / 30
chunk 2 / 30
chunk 3 / 30
chunk 4 / 30
chunk 5 / 30
chunk 6 / 30
chunk 7 / 30
chunk 8 / 30
chunk 9 / 30
chunk 10 / 30
chunk 11 / 30
chunk 12 / 30
chunk 13 / 30
chunk 14 / 30
chunk 15 / 30
chunk 16 / 30
chunk 17 / 30
chunk 18 / 30
chunk 19 / 30
chunk 20 / 30
chunk 21 / 30
chunk 22 / 30
chunk 23 / 30
chunk 24 / 30
chunk 25 / 30
chunk 26 / 30
chunk 27 / 30
chunk 28 / 30
chunk 29 / 30
chunk 30 / 30
reading unwrapPhase in (0, 0, 4268, 700) * 11 ...
use input reference phase
skip pixels with zero/nan value in all interferograms
number of pixels to invert: 2987596 out of 2987600 (100.0%)
inverting network of interferograms into time-series ...
Traceback (most recent call last):
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1116, in <module>
main()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1106, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1050, in run
self.run_network_inversion(sname)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 580, in run_network_inversion
mintpy.ifgram_inversion.main(iargs)
File "/home/insar/tools/MintPy/mintpy/ifgram_inversion.py", line 1139, in main
ifgram_inversion(inps)
File "/home/insar/tools/MintPy/mintpy/ifgram_inversion.py", line 1057, in ifgram_inversion
ts, temp_coh, num_inv_ifg = ifgram_inversion_patch(**data_kwargs)[:-1]
File "/home/insar/tools/MintPy/mintpy/ifgram_inversion.py", line 884, in ifgram_inversion_patch
skip_zero_value=skip_zero_value)
File "/home/insar/tools/MintPy/mintpy/ifgram_inversion.py", line 372, in estimate_timeseries
cond=rcond)[0]
File "/home/insar/anaconda3/envs/isce2/lib/python3.7/site-packages/scipy/linalg/basic.py", line 1152, in lstsq
a1 = _asarray_validated(a, check_finite=check_finite)
File "/home/insar/anaconda3/envs/isce2/lib/python3.7/site-packages/scipy/_lib/_util.py", line 239, in _asarray_validated
a = toarray(a)
File "/home/insar/anaconda3/envs/isce2/lib/python3.7/site-packages/numpy/lib/function_base.py", line 496, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNsMy input file.cfg is:
# vim: set filetype=cfg:
##------------------------ smallbaselineApp.cfg ------------------------##
########## 1. load_data
## a. auto - automatic path pattern for Univ of Miami file structure
## b. load_data.py -H to check more details and example inputs.
## c. compression to save disk usage for ifgramStack.h5 file:
## no - save 0% disk usage, fast [default]
## lzf - save ~57% disk usage, relative slow
## gzip - save ~62% disk usage, very slow [not recommend]
mintpy.load.processor = isce #[isce, aria, snap, gamma, roipac], auto for isce
mintpy.load.updateMode = auto #[yes / no], auto for yes, skip re-loading if HDF5 files are complete
mintpy.load.compression = auto #[gzip / lzf / no], auto for no.
##---------for ISCE only:
mintpy.load.metaFile = ./merged/SLC/*/masterShelve/data.dat #[path of common metadata file for the stack], i.e.: ./master/IW1.xml, ./masterShelve/data.dat
mintpy.load.baselineDir = ./baselines #[path of the baseline dir], i.e.: ./baselines
##---------interferogram datasets:
mintpy.load.unwFile = ./Igrams/*/filt_*.unw #[path pattern of unwrapped interferogram files]
mintpy.load.corFile = ./Igrams/*/filt_*.cor #[path pattern of spatial coherence files]
mintpy.load.connCompFile = ./Igrams/*/filt_*.unw.conncomp #[path pattern of connected components files], optional
mintpy.load.intFile = auto #[path pattern of wrapped interferogram files], optional
mintpy.load.ionoFile = auto #[path pattern of ionospheric delay files], optional
##---------offset datasets (optional):
mintpy.load.azOffFile = auto #[path pattern of azimuth offset file], optional
mintpy.load.rgOffFile = auto #[path pattern of range offset file], optional
mintpy.load.offSnrFile = auto #[path pattern of offset signal-to-noise ratio file], optional
##---------geometry datasets:
mintpy.load.demFile = ./merged/geom_master/hgt.rdr #[path of DEM file]
mintpy.load.lookupYFile = ./merged/geom_master/lat.rdr #[path of latitude /row /y coordinate file], not required for geocoded data
mintpy.load.lookupXFile = ./merged/geom_master/lon.rdr #[path of longitude/column/x coordinate file], not required for geocoded data
mintpy.load.incAngleFile = ./merged/geom_master/los.rdr #[path of incidence angle file], optional
mintpy.load.azAngleFile = ./merged/geom_master/los.rdr #[path of azimuth angle file], optional
mintpy.load.shadowMaskFile = ./merged/geom_master/shadowMask.rdr #[path of shadow mask file], optional
mintpy.load.waterMaskFile = auto #[path of water mask file], optional
mintpy.load.bperpFile = auto #[path pattern of 2D perpendicular baseline file], optional
##---------subset (optional):
## if both yx and lalo are specified, use lalo option unless a) no lookup file AND b) dataset is in radar coord
mintpy.subset.yx = auto #[1800:2000,700:800 / no], auto for no
mintpy.subset.lalo = auto #[31.5:32.5,130.5:131.0 / no], auto for no
########## 2. modify_network
## reference: Yunjun et al. (2019, section 4.2 and 5.3.1)
## 1) Coherence-based network modification = (threshold + MST) by default
## It calculates a average coherence value for each interferogram using spatial coherence and input mask (with AOI)
## Then it finds a minimum spanning tree (MST) network with inverse of average coherence as weight (keepMinSpanTree)
## For all interferograms except for MST's, exclude those with average coherence < minCoherence.
mintpy.network.coherenceBased = yes #[yes / no], auto for no, exclude interferograms with coherence < minCoherence
mintpy.network.keepMinSpanTree = auto #[yes / no], auto for yes, keep interferograms in Min Span Tree network
mintpy.network.minCoherence = auto #[0.0-1.0], auto for 0.7
mintpy.network.maskFile = auto #[file name, no], auto for waterMask.h5 or no [if no waterMask.h5 found]
mintpy.network.aoiYX = auto #[y0:y1,x0:x1 / no], auto for no, area of interest for coherence calculation
mintpy.network.aoiLALO = auto #[lat0:lat1,lon0:lon1 / no], auto for no - use the whole area
## 2) Network modification based on temporal/perpendicular baselines, date etc.
mintpy.network.tempBaseMax = auto #[1-inf, no], auto for no, maximum temporal baseline in days
mintpy.network.perpBaseMax = auto #[1-inf, no], auto for no, maximum perpendicular spatial baseline in meter
mintpy.network.connNumMax = auto #[1-inf, no], auto for no, maximum number of neighbors for each acquisition
mintpy.network.startDate = auto #[20090101 / no], auto for no
mintpy.network.endDate = auto #[20110101 / no], auto for no
mintpy.network.excludeDate = auto #[20080520,20090817 / no], auto for no
mintpy.network.excludeIfgIndex = auto #[1:5,25 / no], auto for no, list of ifg index (start from 0)
mintpy.network.referenceFile = auto #[date12_list.txt / ifgramStack.h5 / no], auto for no
########## 3. reference_point
## Reference all interferograms to one common point in space
## auto - randomly select a pixel with coherence > minCoherence
## however, manually specify using prior knowledge of the study area is highly recommended
## with the following guideline (section 4.3 in Yunjun et al., 2019):
## 1) located in a coherence area
## 2) not affected by strong atmospheric turbulence, i.e. ionospheric streaks
## 3) close to and with similar elevation as the AOI to minimize the impact of spatially correlated atmospheric delay
mintpy.reference.yx = auto #[257,151 / auto]
mintpy.reference.lalo = auto #[31.8,130.8 / auto]
mintpy.reference.maskFile = auto #[filename / no], auto for maskConnComp.h5
mintpy.reference.coherenceFile = auto #[filename], auto for avgSpatialCoh.h5
mintpy.reference.minCoherence = auto #[0.0-1.0], auto for 0.85, minimum coherence for auto method
########## quick_overview
## A quick assessment of:
## 1) possible groud deformation
## using the velocity from the traditional interferogram stacking
## reference: Zebker et al. (1997, JGR)
## 2) distribution of phase unwrapping error
## from the number of interferogram triplets with non-zero integer ambiguity of closue phase
## reference: T_int in Yunjun et al. (2019, CAGEO). Related to section 3.2, equation (8-9) and Fig. 3d-e.
########## 4. correct_unwrap_error (optional)
## connected components (mintpy.load.connCompFile) are required for this step.
## reference: Yunjun et al. (2019, section 3)
## supported methods:
## a. phase_closure - suitable for highly redundant network
## b. bridging - suitable for islands or areas with steep topography
## c. bridging+phase_closure - recommended
mintpy.unwrapError.method = auto #[bridging / phase_closure / bridging+phase_closure / no], auto for no
mintpy.unwrapError.waterMaskFile = auto #[waterMask.h5 / no], auto for waterMask.h5 or no [if no waterMask.h5 found]
## briding options:
## ramp - a phase ramp could be estimated based on the largest reliable region, removed from the entire interferogram
## before estimating the phase difference between reliable regions and added back the correction.
## bridgePtsRadius - half size of the window used to calculate the median value of phase difference
mintpy.unwrapError.ramp = auto #[linear / quadratic], auto for no; recommend linear for L-band data
mintpy.unwrapError.bridgePtsRadius = auto #[1-inf], auto for 50, half size of the window around end points
########## 5. invert_network
## Invert network of interferograms into time-series using weighted least sqaure (WLS) estimator.
## weighting options for least square inversion [fast option available but not best]:
## a. var - use inverse of covariance as weight (Tough et al., 1995; Guarnieri & Tebaldini, 2008) [recommended]
## b. fim - use Fisher Information Matrix as weight (Seymour & Cumming, 1994; Samiei-Esfahany et al., 2016).
## c. coh - use coherence as weight (Perissin & Wang, 2012)
## d. no - uniform weight (Berardino et al., 2002) [fast]
## SBAS (Berardino et al., 2002) = minNormVelocity (yes) + weightFunc (no)
mintpy.networkInversion.weightFunc = auto #[var / fim / coh / no], auto for var
mintpy.networkInversion.waterMaskFile = auto #[filename / no], auto for waterMask.h5 or no [if no waterMask.h5 found]
mintpy.networkInversion.minNormVelocity = auto #[yes / no], auto for yes, min-norm deformation velocity or phase
mintpy.networkInversion.residualNorm = auto #[L2 ], auto for L2, norm minimization solution
## mask options for unwrapPhase of each interferogram before inversion (recommed if weightFunct=no):
## a. coherence - mask out pixels with spatial coherence < maskThreshold
## b. connectComponent - mask out pixels with False/0 value
## c. no - no masking [recommended].
## d. offsetSNR - mask out pixels with offset SNR < maskThreshold [for offset]
mintpy.networkInversion.maskDataset = auto #[coherence / connectComponent / offsetSNR / no], auto for no
mintpy.networkInversion.maskThreshold = auto #[0-inf], auto for 0.4
mintpy.networkInversion.minRedundancy = auto #[1-inf], auto for 1.0, min num_ifgram for every SAR acquisition
## Parallel processing with Dask for HPC
mintpy.networkInversion.parallel = auto #[yes / no], auto for no, parallel processing using dask
mintpy.networkInversion.cluster = auto #[slurm / pbs / lsf], auto for SLURM, cluster type
mintpy.networkInversion.config = auto #[slurm / pbs / lsf / no], auto for no (same as cluster), configuration name
mintpy.networkInversion.numWorker = auto #[int > 0], auto for 40, number of works to deploy
mintpy.networkInversion.walltime = auto #[HH:MM], auto for 00:40, walltime for each dask worker
## Temporal coherence is calculated and used to generate the mask as the reliability measure
## reference: Pepe & Lanari (2006, IEEE-TGRS)
mintpy.networkInversion.minTempCoh = auto #[0.0-1.0], auto for 0.7, min temporal coherence for mask
mintpy.networkInversion.minNumPixel = auto #[int > 0], auto for 100, min number of pixels in mask above
mintpy.networkInversion.shadowMask = auto #[yes / no], auto for yes [if shadowMask is in geometry file] or no.
########## correct_LOD
## Local Oscillator Drift (LOD) correction (for Envisat only)
## reference: Marinkovic and Larsen (2013, Proc. LPS)
## correct LOD if input dataset comes from Envisat
## skip this step for all the other satellites.
########## 6. correct_troposphere (optional and recommended)
## correct tropospheric delay using the following methods:
## a. height_correlation - correct stratified tropospheric delay (Doin et al., 2009, J Applied Geop)
## b. pyaps - use Global Atmospheric Models (GAMs) data (Jolivet et al., 2011; 2014)
## ERA5 - ERA-5 from ECMWF [need to install pyaps3 on GitHub; recommended and turn ON by default]
## ECMWF - ERA-Interim from ECMWF [need to install pyaps on Caltech/EarthDef]
## MERRA - MERRA-2 from NASA [need to install pyaps on Caltech/EarthDef]
## NARR - NARR from NOAA [need to install pyaps on Caltech/EarthDef; recommended for North America]
mintpy.troposphericDelay.method = no #[pyaps / height_correlation / no], auto for pyaps
## Notes for pyaps:
## a. GAM data latency: with the most recent SAR data, there will be GAM data missing, the correction
## will be applied to dates with GAM data available and skipped for the others.
## b. WEATHER_DIR: if you define an environmental variable named WEATHER_DIR to contain the path to a
## directory, then MintPy applications will download the GAM files into the indicated directory. Also MintPy
## application will look for the GAM files in the directory before downloading a new one to prevent downloading
## multiple copies if you work with different dataset that cover the same date/time.
mintpy.troposphericDelay.weatherModel = auto #[ERA5 / ECMWF / MERRA / NARR], auto for ERA5, for pyaps method
mintpy.troposphericDelay.weatherDir = auto #[path2directory], auto for WEATHER_DIR or "./"
## Notes for height_correlation:
## Extra multilooking is applied to estimate the empirical phase/elevation ratio ONLY.
## For an dataset with 5 by 15 looks, looks=8 will generate phase with (5*8) by (15*8) looks
## to estimate the empirical parameter; then apply the correction to original phase (with 5 by 15 looks),
## if the phase/elevation correlation is larger than minCorrelation.
mintpy.troposphericDelay.polyOrder = auto #[1 / 2 / 3], auto for 1, for height_correlation method
mintpy.troposphericDelay.looks = auto #[1-inf], auto for 8, for height_correlation, extra multilooking number
mintpy.troposphericDelay.minCorrelation = 1 #[0.0-1.0], auto for 0, for height_correlation
########## 7. deramp (optional)
## Estimate and remove a phase ramp for each acquisition based on the reliable pixels.
## Recommended for localized deformation signals, i.e. volcanic deformation, landslide and land subsidence, etc.
## NOT recommended for long spatial wavelength deformation signals, i.e. co-, post- and inter-seimic deformation.
mintpy.deramp = linear #[no / linear / quadratic], auto for no - no ramp will be removed
mintpy.deramp.maskFile = auto #[filename / no], auto for maskTempCoh.h5, mask file for ramp estimation
########## 8. correct_topography (optional and recommended)
## Topographic residual (DEM error) correction
## reference: Fattahi and Amelung (2013, IEEE-TGRS)
## stepFuncDate - Specify stepFuncDate option if you know there are sudden displacement jump in your area,
## i.e. volcanic eruption, or earthquake, and check timeseriesStepModel.h5 afterward for their estimation.
## excludeDate - Dates excluded for error estimation only
## pixelwiseGeometry - Use pixel-wise geometry info, such as incidence angle and slant range distance for error estimation
## yes - use pixel-wise geometry when they are available [slow; used by default]
## no - use mean geometry [fast]
mintpy.topographicResidual = auto #[yes / no], auto for yes
mintpy.topographicResidual.polyOrder = auto #[1-inf], auto for 2, poly order of temporal deformation model
mintpy.topographicResidual.phaseVelocity = auto #[yes / no], auto for no - phase, use phase velocity for error estimation
mintpy.topographicResidual.stepFuncDate = auto #[20080529,20100611 / no], auto for no, date of step jump
mintpy.topographicResidual.excludeDate = auto #[20070321 / txtFile / no], auto for exclude_date.txt
mintpy.topographicResidual.pixelwiseGeometry = auto #[yes / no], auto for yes, use pixel-wise geometry info
########## 9.1 residual_RMS (root mean squares for noise evaluation)
## Calculate the Root Mean Square (RMS) of residual phase time-series for each acquisition
## reference: Yunjun et al. (2019, section 4.9 and 5.4)
## To get rid of long wavelength component in space, a ramp is removed for each acquisition
## Set optimal reference date to date with min RMS
## Set exclude dates (outliers) to dates with RMS > cutoff * median RMS (Median Absolute Deviation)
mintpy.residualRMS.maskFile = auto #[file name / no], auto for maskTempCoh.h5, mask for ramp estimation
mintpy.residualRMS.deramp = auto #[quadratic / linear / no], auto for quadratic
mintpy.residualRMS.cutoff = auto #[0.0-inf], auto for 3
########## 9.2 reference_date
## Reference all time-series to one date in time
## reference: Yunjun et al. (2019, section 4.9)
## no - do not change the default reference date (1st date)
mintpy.reference.date = auto #[reference_date.txt / 20090214 / no], auto for reference_date.txt
########## 10. velocity
## Estimate linear velocity from time-series, and from tropospheric delay file if exists.
mintpy.velocity.excludeDate = auto #[exclude_date.txt / 20080520,20090817 / no], auto for exclude_date.txt
mintpy.velocity.startDate = auto #[20070101 / no], auto for no
mintpy.velocity.endDate = auto #[20101230 / no], auto for no
########## 11.1 geocode (post-processing)
mintpy.geocode = auto #[yes / no], auto for yes
mintpy.geocode.SNWE = auto #[-1.2,0.5,-92,-91 / no ], auto for no, output coverage in S N W E in degree
mintpy.geocode.latStep = auto #[0.0-90.0 / None], auto for None, output resolution in degree
mintpy.geocode.lonStep = auto #[0.0-180.0 / None], auto for None - calculate from lookup file
mintpy.geocode.interpMethod = auto #[nearest], auto for nearest, interpolation method
mintpy.geocode.fillValue = auto #[np.nan, 0, ...], auto for np.nan, fill value for outliers.
########## 11.2 google_earth (post-processing)
mintpy.save.kmz = yes #[yes / no], auto for yes, save geocoded velocity to Google Earth KMZ file
########## 11.3 hdfeos5 (post-processing)
mintpy.save.hdfEos5 = auto #[yes / no], auto for no, save time-series to HDF-EOS5 format
mintpy.save.hdfEos5.update = auto #[yes / no], auto for no, put XXXXXXXX as endDate in output filename
mintpy.save.hdfEos5.subset = auto #[yes / no], auto for no, put subset range info in output filename
########## 11.4 plot
mintpy.plot = auto #[yes / no], auto for yes, plot files generated by default processing to pic folderThanks for your help
Pedro Espin
yunjunz
commented
4 years ago Have you updated your mintpy code? With the commit tagged here previously, L should not be 0 anymore, in any circumstance.
alejobeap
commented
4 years ago Dear Yes I updated the last version of mintpy I run again.
I have the error:
******************** step - correct_topography ********************
dem_error.py timeseries_ramp.h5 -t /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/smallbaselineApp.cfg -o timeseries_ramp_demErr.h5 --update -g /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/geometryRadar.h5
read options from template file: smallbaselineApp.cfg
--------------------------------------------------
update mode: ON
1) output file timeseries_ramp_demErr.h5 NOT found.
run or skip: run.
open geometry file: geometryRadar.h5
read 2D incidenceAngle,slantRangeDistance from geometry file: geometryRadar.h5
read mean bperp from timeseries file
--------------------------------------------------------------------------------
correct topographic phase residual (DEM error) (Fattahi & Amelung, 2013, IEEE-TGRS)
ordinal least squares (OLS) inversion with L2-norm minimization on: phase (pixel-wisely)
temporal deformation model: polynomial order = 2
--------------------------------------------------------------------------------
open timeseries file: timeseries_ramp.h5
reading timeseries data from file: timeseries_ramp.h5 ...
inverting DEM error ...
skip pixels with ZERO in ALL acquisitions
skip pixels with NaN in ANY acquisitions
skip pixels with ZERO / NaN value in incidenceAngle / slantRangeDistance
Traceback (most recent call last):
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1116, in <module>
main()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1106, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1062, in run
self.run_topographic_residual_correction(sname)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 841, in run_topographic_residual_correction
mintpy.dem_error.main(iargs)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 483, in main
inps = correct_dem_error(inps, A_def)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 365, in correct_dem_error
mask *= geom_data != 0.
ValueError: operands could not be broadcast together with shapes (3514698,) (84391540,) (3514698,) Thanks Pedro
yunjunz
commented
4 years ago The previous step invert_network has passed, this is good news.
For this step, it's a size mismatch between the time-series file and the geometry file, as you can see it from the error message. I guess you used the full resolution geometry file, you could try to change your template file from:
mintpy.load.demFile = ./merged/geom_master/hgt.rdr #[path of DEM file]to
mintpy.load.demFile = ./geom_master/hgt.rdr #[path of DEM file]As shown in https://mintpy.readthedocs.io/en/latest/dir_structure/, for all mintpy.load.* options.
alejobeap
commented
4 years ago Dear I don't have the same structure after producing the stack at ISCE. The geom_master folder is inside the merged folder.
I have also run again but with this option in auto
mintpy.networkInversion.waterMaskFile = auto #[filename / no], auto for waterMask.h5 or no [if no waterMask.h5 found]
mintpy.networkInversion.shadowMask = auto #[yes / no], auto for yes [if shadowMask is in geometry file] or no.and I get this error:
******************** step - invert_network ********************
ifgram_inversion.py /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/ifgramStack.h5 -t /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/smallbaselineApp.cfg --update
use dataset "unwrapPhase" by default
update mode: ON
1) output files already exist: ['timeseries.h5', 'temporalCoherence.h5', 'numInvIfgram.h5'].
2) output dataset is newer than input dataset: unwrapPhase.
3) all key configuration parameters are the same: ['obsDatasetName', 'numIfgram', 'weightFunc', 'maskDataset', 'maskThreshold', 'minRedundancy', 'minNormVelocity'].
run or skip: skip.
generate_mask.py temporalCoherence.h5 -m 0.7 -o maskTempCoh.h5 --base /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/geometryRadar.h5 --base-dataset shadowMask --base-value 1
update mode: ON
1) output file: maskTempCoh.h5 already exists and newer than input file: temporalCoherence.h5
2) NOT all key configration parameters are the same: ['mintpy.networkInversion.minTempCoh', 'mintpy.networkInversion.shadowMask']
run or skip: run
input temporalCoherence file: temporalCoherence.h5
read temporalCoherence.h5
create initial mask with the same size as the input file and all = 1
all pixels with nan value = 0
exclude pixels with value < 0.7
Traceback (most recent call last):
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1116, in <module>
main()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1106, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1050, in run
self.run_network_inversion(sname)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 583, in run_network_inversion
self.generate_temporal_coherence_mask()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 621, in generate_temporal_coherence_mask
mintpy.generate_mask.main(iargs)
File "/home/insar/tools/MintPy/mintpy/generate_mask.py", line 281, in main
inps.outfile = create_threshold_mask(inps)
File "/home/insar/tools/MintPy/mintpy/generate_mask.py", line 219, in create_threshold_mask
mask[base_data == float(inps.base_value)] = 0
IndexError: boolean index did not match indexed array along dimension 0; dimension is 1647 but corresponding boolean dimension is 6590When you put it NO it does not run until this error again:
--------------------------------------------------
mintpy.networkInversion.waterMaskFile = no #[filename / no], auto for waterMask.h5 or no [if no waterMask.h5 found]
mintpy.networkInversion.shadowMask = no #[yes / no], auto for yes [if shadowMask is in geometry file] or no.
******************** step - deramp ********************
Remove for each acquisition a phase ramp: linear
remove_ramp.py timeseries.h5 -s linear -m maskTempCoh.h5 -o timeseries_ramp.h5 --update
update mode: ON
1) output file timeseries_ramp.h5 NOT found.
run or skip: run.
remove linear ramp from file: timeseries.h5
read mask file: maskTempCoh.h5
reading data ...
estimating phase ramp ...
create HDF5 file: timeseries_ramp.h5 with w mode
create dataset /timeseries of float32 in size of (12, 1647, 2134) with compression=None
create dataset /bperp of float32 in size of (12,) with compression=None
create dataset /date of |S8 in size of (12,) with compression=None
finished writing to timeseries_ramp.h5
time used: 00 mins 1.6 secs.
add/update the following configuration metadata to file:
['mintpy.deramp', 'mintpy.deramp.maskFile']
******************** step - correct_topography ********************
dem_error.py timeseries_ramp.h5 -t /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/smallbaselineApp.cfg -o timeseries_ramp_demErr.h5 --update -g /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/geometryRadar.h5
read options from template file: smallbaselineApp.cfg
update mode: ON
1) output file timeseries_ramp_demErr.h5 NOT found.
run or skip: run.
open geometry file: geometryRadar.h5
read 2D incidenceAngle,slantRangeDistance from geometry file: geometryRadar.h5
read mean bperp from timeseries file
correct topographic phase residual (DEM error) (Fattahi & Amelung, 2013, IEEE-TGRS)
ordinal least squares (OLS) inversion with L2-norm minimization on: phase (pixel-wisely)
temporal deformation model: polynomial order = 2
open timeseries file: timeseries_ramp.h5
reading timeseries data from file: timeseries_ramp.h5 ...
inverting DEM error ...
skip pixels with ZERO in ALL acquisitions
skip pixels with NaN in ANY acquisitions
skip pixels with ZERO / NaN value in incidenceAngle / slantRangeDistance
Traceback (most recent call last):
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1116, in <module>
main()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1106, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1062, in run
self.run_topographic_residual_correction(sname)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 841, in run_topographic_residual_correction
mintpy.dem_error.main(iargs)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 483, in main
inps = correct_dem_error(inps, A_def)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 365, in correct_dem_error
mask *= geom_data != 0.
ValueError: operands could not be broadcast together with shapes (3514698,) (84391540,) (3514698,) mintpy.networkInversion.waterMaskFile = auto #[filename / no], auto for waterMask.h5 or no [if no waterMask.h5 found]
mintpy.networkInversion.shadowMask = no #[yes / no], auto for yes [if shadowMask is in geometry file] or no.the same error:
******************** step - correct_topography ********************
dem_error.py timeseries_ramp.h5 -t /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/smallbaselineApp.cfg -o timeseries_ramp_demErr.h5 --update -g /mnt/procesamiento/Procesamiento_TX/Sangay/Stack_Descendente/inputs/geometryRadar.h5
read options from template file: smallbaselineApp.cfg
update mode: ON
1) output file timeseries_ramp_demErr.h5 NOT found.
run or skip: run.
open geometry file: geometryRadar.h5
read 2D incidenceAngle,slantRangeDistance from geometry file: geometryRadar.h5
read mean bperp from timeseries file
correct topographic phase residual (DEM error) (Fattahi & Amelung, 2013, IEEE-TGRS)
ordinal least squares (OLS) inversion with L2-norm minimization on: phase (pixel-wisely)
temporal deformation model: polynomial order = 2
open timeseries file: timeseries_ramp.h5
reading timeseries data from file: timeseries_ramp.h5 ...
inverting DEM error ...
skip pixels with ZERO in ALL acquisitions
skip pixels with NaN in ANY acquisitions
skip pixels with ZERO / NaN value in incidenceAngle / slantRangeDistance
Traceback (most recent call last):
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1116, in <module>
main()
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1106, in main
app.run(steps=inps.runSteps, plot=inps.plot)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 1062, in run
self.run_topographic_residual_correction(sname)
File "/home/insar/tools/MintPy/mintpy/smallbaselineApp.py", line 841, in run_topographic_residual_correction
mintpy.dem_error.main(iargs)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 483, in main
inps = correct_dem_error(inps, A_def)
File "/home/insar/tools/MintPy/mintpy/dem_error.py", line 365, in correct_dem_error
mask *= geom_data != 0.
ValueError: operands could not be broadcast together with shapes (3514698,) (84391540,) (3514698,) Thank you very much for your help.
yunjunz
commented
4 years ago The file/dir structure should be the same as shown in the link above. This seems more of the issue in the ISCE-2 stripmapStack processing. You might want to search / ask it in the ISCE forum.
alejobeap
commented
4 years ago Thanks for your help I'll ask in the ISCE forum I used stackStripMap.py
Pedro
I processed with ISCE some TerrasarX images at the time of using Mintpy is generating the error in the invert network
Full script that generated the error
Full error message
System information