search

yumorishita / LiCSBAS

LiCSBAS: InSAR time series analysis package using LiCSAR products
https://doi.org/10.3390/RS12030424
GNU General Public License v3.0
224 stars 109 forks source link

The processing stucked in Step 1-6 #100

Closed CanZhang-zc closed 3 years ago

CanZhang-zc commented 3 years ago

Dear Yu, Please, the step 1-6 processing stucked again with the n_para=1 and the smaller filtwidth_km=1. Then I updated the code of 1-6, but the processing runs directly to the end, and there is no 16 Successfully finished!!. How to solve this problem now? Sorry for any inconvenience caused, your assistance will be highly appreciated.

Here is the log file: LiCSBAS16_filt_ts.py ver1.5.1 20210311 Y. Morishita LiCSBAS16_filt_ts.py -t TS_GEOCml1 --n_para 1 --ex_range 0:0/0:0

Size of image (w,l) : 3106, 2691 Number of images : 114 Width of filter in space : 2 km (21.2x18.0 pixel) Width of filter in time : 0.118 yr (43 days) Deramp flag : [] hgt-linear flag : False Excluded range : 0:3106/0:2691

HP filter in time, LP filter in space, 0/114th image... 10/114th image... 20/114th image... 30/114th image... 40/114th image... 50/114th image... 60/114th image... 70/114th image... 80/114th image... 90/114th image... 100/114th image... 110/114th image...

Create png for increment with 1 parallel processing... /home/zhangcan/anaconda3/envs/licsbas/lib/python3.6/site-packages/matplotlib/backends/qt_editor/figureoptions.py:11: MatplotlibDeprecationWarning: The support for Qt4 was deprecated in Matplotlib 3.3 and will be removed two minor releases later. from matplotlib.backends.qt_compat import QtGui

Find stable reference point... Selected ref: 1120:1121/1022:1023

Calculate velocity of filtered time series... First, solving 4724068/4866500th points with full cum...

yumorishita commented 3 years ago

You excluded all pixels for the filtering by --ex_range 0:0/0:0 and this means no effect is made by the filtering. Change the parameter of ex_range.

CanZhang-zc commented 3 years ago

Thanks a lot. I will change the parameter of ex_range first. If filtering the entire area, is it correct to use the default parameters? In addition, I would like to ask if this parameter error is the cause of this step's stuck?

yumorishita commented 3 years ago

Yes, the default blank parameter means filtering the entire area. I am not sure about the cause of the stuck. Just try.

CanZhang-zc commented 3 years ago

Thank you so much. I will try it first.

CanZhang-zc commented 3 years ago

Dear Yu, I have another question to ask. If I want to process the entire area, then step 0-5 are not executed, will it affect the quality of the final results?

yumorishita commented 3 years ago

Yes, you may get a different result.

CanZhang-zc commented 3 years ago

Thank you so much. But, if I want to process the entire area, should I perform the clip of the entire area in step 0-5?

yumorishita commented 3 years ago

No, clipping the entire area in step05 has no meaning but the waste of time and disk space and you would get the same result as just skipping step05. I meant that you may get different results from the entire one if you clip an area in step05.

CanZhang-zc commented 3 years ago

Thank you so much, I understand what you mean now. And I want to ask if step 1-6 still appears stuck, clipping twice in step 0-5 to combine into a complete area to avoid stuck and get the results of the entire area...Is the way feasible?

CanZhang-zc commented 3 years ago

And this time, step 1-3 first stucked just now. But 1-3 did not stuck before... If now I run it again directly from 1-3, will the batch from the last time it stucked? Or reprocess 1-3 all?

Here is the log file: LiCSBAS13_sb_inv.py ver1.5.2 20210311 Y. Morishita LiCSBAS13_sb_inv.py -d GEOCml1mask --n_para 1 --n_para 1

In geographical coordinates

Not enough memory available compared to mem_size (8000 MB). Reduce mem_size automatically to 7373 MB. Read cum data on memory (fast but need memory).

Size of image (w,l) : 3106, 2691 of all images : 115 of images to be used : 114 of all ifgs : 413 of ifgs to be used : 375 of removed ifgs : 38 Threshold of used unw : 114

Reference area (X/Y) : 1393:1394/711:712 Allowed memory size : 7373 MB Number of patches : 9 Inversion algorism : LS Gamma value : 0.0001

Process 299/2691th line (1/9th patch)... Reading 375 ifg's unw data... 755694/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... /home/zhangcan/anaconda3/envs/licsbas/lib/python3.6/site-packages/matplotlib/backends/qt_editor/figureoptions.py:11: MatplotlibDeprecationWarning: The support for Qt4 was deprecated in Matplotlib 3.3 and will be removed two minor releases later. from matplotlib.backends.qt_compat import QtGui Running 1/ 1th patch...

Small Baseline inversion by LS...

Solving 47507/173000th points with full unw at a time... Next, solve 125493 points including nan point-by-point... Running 125000/125493th point... Elapsed time for 1th patch: 00h 09m 54s

Process 598/2691th line (2/9th patch)... Reading 375 ifg's unw data... 177553/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... Running 1/ 1th patch...

Small Baseline inversion by LS...

Solving 140632/751141th points with full unw at a time... Next, solve 610509 points including nan point-by-point... Running 610000/610509th point... Elapsed time for 2th patch: 00h 46m 42s

Process 897/2691th line (3/9th patch)... Reading 375 ifg's unw data... 134906/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... Running 1/ 1th patch...

Small Baseline inversion by LS...

Solving 182580/793788th points with full unw at a time... Next, solve 611208 points including nan point-by-point... Running 611000/611208th point... Elapsed time for 3th patch: 00h 48m 22s

Process 1196/2691th line (4/9th patch)... Reading 375 ifg's unw data... 140696/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... Running 1/ 1th patch...

Small Baseline inversion by LS...

Solving 116630/787998th points with full unw at a time... Next, solve 671368 points including nan point-by-point... Running 671000/671368th point... Elapsed time for 4th patch: 00h 52m 21s

Process 1495/2691th line (5/9th patch)... Reading 375 ifg's unw data... 141165/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... Running 1/ 1th patch...

Small Baseline inversion by LS...

Solving 151937/787529th points with full unw at a time... Next, solve 635592 points including nan point-by-point... Running 635000/635592th point... Elapsed time for 5th patch: 00h 49m 44s

Process 1794/2691th line (6/9th patch)... Reading 375 ifg's unw data... 163628/928694 points removed due to not enough ifg data...

Identifing gaps, and counting n_gap and n_ifg_noloop, with 1 parallel processing... Running 1/ 1th patch...

yumorishita commented 3 years ago

Could you tell me:

CanZhang-zc commented 3 years ago

Thanks Yu. I'll tell you these information below. But some I'm not sure. As a supplement, the desktop hard drive that I use is SEAGATE(ST4000DM004) BarraCuda 4TB 5000rpm 256MB(Cache), and the computer that I use is Lenovo Y7000(2020) i7-10750H 16G 512G(SSD) GTX1650-4G(Discrete graphics card).

Frame ID: 099A_05815_131313 What is your deformation of interest? (e.g., subsidence, landslide, volcano): subsidence in large area Memory size: 4GB Storage size: 16G Storage type (e.g., HDD or SSD, USB or SATA): (Does this refer to desktop hard drive?) SATA Interface OS: Ubuntu 20.04 (Win10 as a dual OS)

yumorishita commented 3 years ago

Sorry, but I have no idea of the stuck. Have you tried several times and always stop at the same line?

The target subsidence area may be large, but it is not an entire frame, right? I generally recommend clipping the area of interest, because the processing would be much faster and the quality would be better.

CanZhang-zc commented 3 years ago

OK, Thanks Yu. I'll try it again to see if it always stop at the same line every times. But the goal of my research is to get the surface deformation of a large area for about 5 years (the whole may be my interest), so multiple frames may be needed... So I want to ask clipping twice in step 0-5 to combine into a complete area to avoid stuck and get the results of the entire area...Is the way feasible?

yumorishita commented 3 years ago

I recommend

  1. process the entre frame with a large nlook (e.g., 4 or 10) in step02 and finish to step16
  2. identify areas with significant deformation from the low resolution result
  3. clip the deforming areas in step05 with nlook=1

to efficiently get the high resolution deformation data.

CanZhang-zc commented 3 years ago

Thank you so much. I'll first run again to see if it always stop at the same line and try all these above later. In addition, I would like to ask whether there are other parameters that need to be adjusted in the parameter configuration. Do they generally use the default values?

yumorishita commented 3 years ago

Yes, I generally use the default values, but sometimes parameter adjustment is needed to improve the quality. It highly depends on the situation and I cannot make a general comment.

CanZhang-zc commented 3 years ago

Thanks, I understand. I will reprocess later, thank you again.

CanZhang-zc commented 3 years ago

Dear Yu, Is the nlook in step 0-2 usually using 4 or 10? What about others?

yumorishita commented 3 years ago

The default is 1. It could be any number but I usually use 10 for a quick look.

CanZhang-zc commented 3 years ago

Thank you so much. I get it. I will reprocess.

CanZhang-zc commented 3 years ago

Dear Yu, I want to ask a simple question. For the LiCSAR products downloaded in step 0-1, can I manually delete some products as needed to adjust the time baseline length (e.g., 12d/24d/36d)? Sorry for any inconvenience caused, your assistance will be highly appreciated.

yumorishita commented 3 years ago

Yes, just move the yyyymmdd_yyyymmdd directories from the GEOC directory to somewhere.

CanZhang-zc commented 3 years ago

Thank you so much Yu. I get it. In this case, do I only need to make 2 and 3 the same continuously (e.g., yyyymmdd1_yyyymmdd2 and yyyymmdd3_yyyymmdd4)?

yumorishita commented 3 years ago

Sorry, I don't understand what you mean.

CanZhang-zc commented 3 years ago

Sorry Yu, I didn't describe my question clearly. I mean if the time baseline is 24d, then the directories should be adjusted to “20180315_20180408, 20180408_20180502......”. Is this move right?

yumorishita commented 3 years ago

I think so.

CanZhang-zc commented 3 years ago

Thank you so much. I get it.

CanZhang-zc commented 3 years ago

Dear Yu, After I adjusted the time baseline length, I met this error. Why it happens and how to solve it? Sorry for any inconvenience caused, your assistance will be highly appreciated.

Here is the log file: LiCSBAS12_loop_closure.py ver1.6 20210311 Y. Morishita LiCSBAS12_loop_closure.py -d GEOCml1GACOSmask --multi_prime

loop_thre : 1.5 rad

1st Loop closure check and make png for all possible 0 loops, with 0 parallel processing... /home/zhangcan/anaconda3/envs/licsbas/lib/python3.6/site-packages/matplotlib/backends/qt_editor/figureoptions.py:11: MatplotlibDeprecationWarning: The support for Qt4 was deprecated in Matplotlib 3.3 and will be removed two minor releases later. from matplotlib.backends.qt_compat import QtGui Traceback (most recent call last): File "/home/zhangcan/LiCSBAS/bin/LiCSBAS12_loop_closure.py", line 869, in sys.exit(main()) File "/home/zhangcan/LiCSBAS/bin/LiCSBAS12_loop_closure.py", line 295, in main p = q.Pool(_n_para) File "/home/zhangcan/anaconda3/envs/licsbas/lib/python3.6/multiprocessing/context.py", line 119, in Pool context=self.get_context()) File "/home/zhangcan/anaconda3/envs/licsbas/lib/python3.6/multiprocessing/pool.py", line 167, in init raise ValueError("Number of processes must be at least 1") ValueError: Number of processes must be at least 1

CanZhang-zc commented 3 years ago

And here is the settings: start_step="02" # 01-05, 11-16 end_step="16" # 01-05, 11-16

nlook="1" # multilook factor, used in step02 GEOCmldir="GEOCml${nlook}" # If start from 11 or later after doing 03-05, use e.g., GEOCml${nlook}GACOSmaskclip n_para="" # Number of paralell processing in step 02-05,12,13,16. default: number of usable CPU gpu="n" # y/n check_only="n" # y/n. If y, not run scripts and just show commands to be done

logdir="log" log="$logdir/$(date +%Y%m%d%H%M)$(basename $0 .sh)_${startstep}${end_step}.log"

Optional steps (03-05) order_op03_05="03 04 05" # can change order e.g., 05 03 04 do03op_GACOS="y" # y/n do04op_mask="y" # y/n do05op_clip="n" # y/n p04_mask_coh_thre="0.1" # e.g. 0.2 p04_mask_range="" # e.g. 10:100/20:200 (ix start from 0) p04_mask_range_file="" # Name of file containing range list p05_clip_range="" # e.g. 10:100/20:200 (ix start from 0) p05_clip_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg)

Frequently used options. If blank, use default. p01_start_date="20180315" # default: 20141001 p01_end_date="20210311" # default: today p01_get_gacos="y" # y/n p11_unw_thre="" # default: 0.3 p11_coh_thre="" # default: 0.05 p12_loop_thre="" # default: 1.5 rad p12_multi_prime="y" # y/n. y recommended p12_rm_ifg_list="" # List file containing ifgs to be manually removed p15_coh_thre="" # default: 0.05 p15_n_unw_r_thre="" # default: 1.5 p15_vstd_thre="" # default: 100 mm/yr p15_maxTlen_thre="" # default: 1 yr p15_n_gap_thre="" # default: 10 p15_stc_thre="" # default: 5 mm p15_n_ifg_noloop_thre="" # default: 50 p15_n_loop_err_thre="" # default: 5 p15_resid_rms_thre="" # default: 2 mm p16_filtwidth_km="" # default: 2 km p16_filtwidth_yr="" # default: avg_interval*3 yr p16_deg_deramp="" # 1, bl, or 2. default: no deramp p16_hgt_linear="n" # y/n. default: n p16_hgt_min="" # default: 200 (m) p16_hgt_max="" # default: 10000 (m) p16_range="" # e.g. 10:100/20:200 (ix start from 0) p16_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg) p16_ex_range="" # e.g. 10:100/20:200 (ix start from 0) p16_ex_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg)

Less frequently used options. If blank, use default. p01_frame="128A_05969_131313" # e.g. 021D_04972_131213 p01_n_para="" # default: 4 p02_GEOCdir="" # default: GEOC p02_GEOCmldir="" # default: GEOCml$nlook p02_freq="" # default: 5.405e9 Hz p02_n_para="" # default: # of usable CPU p03_inGEOCmldir="" # default: $GEOCmldir p03_outGEOCmldir_suffix="" # default: GACOS p03_fillhole="y" # y/n. default: n p03_gacosdir="" # default: GACOS p03_n_para="" # default: # of usable CPU p04_inGEOCmldir="" # default: $GEOCmldir p04_outGEOCmldir_suffix="" # default: mask p04_n_para="" # default: # of usable CPU p05_inGEOCmldir="" # default: $GEOCmldir p05_outGEOCmldir_suffix="" # default: clip p05_n_para="" # default: # of usable CPU p11_GEOCmldir="" # default: $GEOCmldir p11TSdir="" # default: TS$GEOCmldir p12_GEOCmldir="" # default: $GEOCmldir p12TSdir="" # default: TS$GEOCmldir p12_n_para="" # default: # of usable CPU p13_GEOCmldir="" # default: $GEOCmldir p13TSdir="" # default: TS$GEOCmldir p13_inv_alg="" # LS (default) or WLS p13_mem_size="" # default: 8000 (MB) p13_gamma="" # default: 0.0001 p13_n_para=1 # default: # of usable CPU p13_n_unw_r_thre="" # defualt: 1 p13_keep_incfile="n" # y/n. default: n p14TSdir="" # default: TS$GEOCmldir p14_mem_size="" # default: 4000 (MB) p15TSdir="" # default: TS$GEOCmldir p15_vmin="" # default: auto (mm/yr) p15_vmax="" # default: auto (mm/yr) p15_keep_isolated="n" # y/n. default: n p15_noautoadjust="n" # y/n. default: n p16TSdir="" # default: TS$GEOCmldir p16_nomask="n" # y/n. default: n p16_n_para=1 # default: # of usable CPU

CanZhang-zc commented 3 years ago

I just find the error happens because of the moves. But I don't know if moves, how to avoid it.

yumorishita commented 3 years ago

LiCSBAS assumes a redundant network of interferograms (i.e., loops of interferograms can be created). I recommend, e.g., adding 20180315_20180502 if you remain 20180315_20180408 and 20180408_20180502 interferograms to create a loop.

CanZhang-zc commented 3 years ago

Thank you so much. I understood it just now. And I want to ask another question. What differences (e.g., the precision) of results between the one loop and the loops (e.g., 4)?

yumorishita commented 3 years ago

Generally the more redundant, the better precision.

CanZhang-zc commented 3 years ago

vel filt mskd Dear Yu, I have finished the whole process, and not move any directories. But the result does not look good, is there a problem with the parameter setting?

CanZhang-zc commented 3 years ago

Here is the setting: start_step="02" # 01-05, 11-16 end_step="16" # 01-05, 11-16

nlook="1" # multilook factor, used in step02 GEOCmldir="GEOCml${nlook}" # If start from 11 or later after doing 03-05, use e.g., GEOCml${nlook}GACOSmaskclip n_para="" # Number of paralell processing in step 02-05,12,13,16. default: number of usable CPU gpu="n" # y/n check_only="n" # y/n. If y, not run scripts and just show commands to be done

logdir="log" log="$logdir/$(date +%Y%m%d%H%M)$(basename $0 .sh)_${startstep}${end_step}.log"

Optional steps (03-05) order_op03_05="03 04 05" # can change order e.g., 05 03 04 do03op_GACOS="y" # y/n do04op_mask="y" # y/n do05op_clip="n" # y/n p04_mask_coh_thre="0.1" # e.g. 0.2 p04_mask_range="" # e.g. 10:100/20:200 (ix start from 0) p04_mask_range_file="" # Name of file containing range list p05_clip_range="" # e.g. 10:100/20:200 (ix start from 0) p05_clip_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg)

Frequently used options. If blank, use default. p01_start_date="20180315" # default: 20141001 p01_end_date="20210311" # default: today p01_get_gacos="y" # y/n p11_unw_thre="" # default: 0.3 p11_coh_thre="" # default: 0.05 p12_loop_thre="" # default: 1.5 rad p12_multi_prime="y" # y/n. y recommended p12_rm_ifg_list="" # List file containing ifgs to be manually removed p15_coh_thre="" # default: 0.05 p15_n_unw_r_thre="" # default: 1.5 p15_vstd_thre="" # default: 100 mm/yr p15_maxTlen_thre="" # default: 1 yr p15_n_gap_thre="" # default: 10 p15_stc_thre="" # default: 5 mm p15_n_ifg_noloop_thre="" # default: 50 p15_n_loop_err_thre="" # default: 5 p15_resid_rms_thre="" # default: 2 mm p16_filtwidth_km="" # default: 2 km p16_filtwidth_yr="" # default: avg_interval*3 yr p16_deg_deramp="" # 1, bl, or 2. default: no deramp p16_hgt_linear="n" # y/n. default: n p16_hgt_min="" # default: 200 (m) p16_hgt_max="" # default: 10000 (m) p16_range="" # e.g. 10:100/20:200 (ix start from 0) p16_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg) p16_ex_range="" # e.g. 10:100/20:200 (ix start from 0) p16_ex_range_geo="" # e.g. 130.11/131.12/34.34/34.6 (in deg)

Less frequently used options. If blank, use default. p01_frame="128A_05969_131313" # e.g. 021D_04972_131213 p01_n_para="" # default: 4 p02_GEOCdir="" # default: GEOC p02_GEOCmldir="" # default: GEOCml$nlook p02_freq="" # default: 5.405e9 Hz p02_n_para="" # default: # of usable CPU p03_inGEOCmldir="" # default: $GEOCmldir p03_outGEOCmldir_suffix="" # default: GACOS p03_fillhole="y" # y/n. default: n p03_gacosdir="" # default: GACOS p03_n_para="" # default: # of usable CPU p04_inGEOCmldir="" # default: $GEOCmldir p04_outGEOCmldir_suffix="" # default: mask p04_n_para="" # default: # of usable CPU p05_inGEOCmldir="" # default: $GEOCmldir p05_outGEOCmldir_suffix="" # default: clip p05_n_para="" # default: # of usable CPU p11_GEOCmldir="" # default: $GEOCmldir p11TSdir="" # default: TS$GEOCmldir p12_GEOCmldir="" # default: $GEOCmldir p12TSdir="" # default: TS$GEOCmldir p12_n_para="" # default: # of usable CPU p13_GEOCmldir="" # default: $GEOCmldir p13TSdir="" # default: TS$GEOCmldir p13_inv_alg="" # LS (default) or WLS p13_mem_size="" # default: 8000 (MB) p13_gamma="" # default: 0.0001 p13_n_para=1 # default: # of usable CPU p13_n_unw_r_thre="" # defualt: 1 p13_keep_incfile="n" # y/n. default: n p14TSdir="" # default: TS$GEOCmldir p14_mem_size="" # default: 4000 (MB) p15TSdir="" # default: TS$GEOCmldir p15_vmin="" # default: auto (mm/yr) p15_vmax="" # default: auto (mm/yr) p15_keep_isolated="n" # y/n. default: n p15_noautoadjust="n" # y/n. default: n p16TSdir="" # default: TS$GEOCmldir p16_nomask="n" # y/n. default: n p16_n_para=1 # default: # of usable CPU

yumorishita commented 3 years ago

The result looks good for me. What and how to improve the result highly depends on your target (i.e., what you want to detect) and therefore I cannot make a general comment. To improve your result yourself, I recommend carefully reading my papers below, understanding the theoretical background of the LiCSBAS processing, checking all images in the output of LiCSBAS, and trying tweaking parameters many times.

Morishita, Y.; Lazecky, M.; Wright, T.J.; Weiss, J.R.; Elliott, J.R.; Hooper, A. LiCSBAS: An Open-Source InSAR Time Series Analysis Package Integrated with the LiCSAR Automated Sentinel-1 InSAR Processor. Remote Sens. 2020, 12, 424, https://doi.org/10.3390/RS12030424.

Morishita, Y.: Nationwide urban ground deformation monitoring in Japan using Sentinel-1 LiCSAR products and LiCSBAS. Prog. Earth Planet. Sci. 2021, 8, 6, https://doi.org/10.1186/s40645-020-00402-7.

CanZhang-zc commented 3 years ago

Thank you very mush Yu, I get it. Now I'll close the issue. Thank you for your response and help these days. If I have other questions, I'll open another new issue. Wish you all the best!