LiCSBAS12_loop_closure.py

[oguzhannysr]

Hello @yumorishita, the computer gets stuck at this step. What is the reason for this? I do it without cropping the area. 202409050828batch_LiCSBAS_01_16.log

[yumorishita]

Try n_para=1. https://github.com/yumorishita/LiCSBAS/blob/5db6516a5a9bad31445d916e61e7ee03c3daa1fd/batch_LiCSBAS.sh#L24

[oguzhannysr]

@yumorishita , I try by setting n12_para to 1

[oguzhannysr]

@yumorishita , Now I'm stuck here, which n_para should I change?

  Running  64000/ 65117th point...
  Running  63000/ 65117th point...
  Running  61000/ 65117th point...
  Running  62000/ 65117th point...
  Running  65000/ 65117th point...
  Running  60000/ 65117th poi
```nt...
  Elapsed time for 11th patch: 00h 05m 58s

Process 624/2689th line (12/52th patch)...
  Reading 724 ifg's unw data...
  84278/183768 points removed due to not enough ifg data...

  Identifing gaps, and counting n_gap and n_ifg_noloop,
  with 16 parallel processing...
    Running  1/16th patch...
    Running  2/16th patch...
    Running  3/16th patch...
    Running  4/16th patch...
    Running  5/16th patch...
    Running  6/16th patch...
    Running  9/16th patch...
    Running 10/16th patch...
    Running 11/16th patch...
    Running 12/16th patch...
    Running 14/16th patch...
    Running 15/16th patch...
    Running 16/16th patch...
    Running  8/16th patch...
    Running  7/16th patch...
    Running 13/16th patch...

[yumorishita]

I already mentioned above.

[oguzhannysr]

@yumorishita,

202409052217batch_LiCSBAS_01_16.log

Process 2673/2689th line (243/251th patch)...
  Reading 724 ifg's unw data...
  38874/38874 points removed due to not enough ifg data...
  Elapsed time for 243th patch: 00h 05m 04s

Process 2684/2689th line (244/251th patch)...
  Reading 724 ifg's unw data...
  38874/38874 points removed due to not enough ifg data...
  Elapsed time for 244th patch: 00h 00m 57s

Process 269
5/2689th line (245/251th patch)...
  Reading 724 ifg's unw data...
Traceback (most recent call last):
  File "/home/yasar/LICSBAS/bin/LiCSBAS13_sb_inv.py", line 996, in <module>
    sys.exit(main())
  File "/home/yasar/LICSBAS/bin/LiCSBAS13_sb_inv.py", line 541, in main
    unw = np.fromfile(f, dtype=np.float32, count=countl).reshape((lengththis, width))*coef_r2m
ValueError: cannot reshape array of size 17670 into shape (11,3534)

[yumorishita]

Maybe some of the GEOCml1//unw files are corrupted. Please check all the files have the same size.

[oguzhannysr]

@yumorishita , There are more than 1000 folders, how can I do this more easily? Also, if there is a corrupt file, what should I do? Should I go back to the beginning?

[yumorishita]

Remove corrupted files and restart from step11.

[oguzhannysr]

@yumorishita , I checked for corrupt files, but all files were the same size and as they should be. I don't know what to do now?

[yumorishita]

I have no idea, maybe some other files are corrupted. What I can suggest is removing all the files and rerunning from the start.

[oguzhannysr]

thank you...

[oguzhannysr]

@yumorishita , Dear professor, I have a different technical question for you apart from this topic. Using LICSBAS, I produced cumulative deformation los maps in ascending and descending orbits (2016-2024). How technically correct is it to separate each image pair as east-west and up-down using the cumulative los data produced for the ascending and descending orbits as of 2016? Because after a certain year (failure of one of the Sentinel-1 satellite constellations), the ascending and descending image dates do not match. I need your opinion on this matter.. up-down folder east-west folder up-down timeseries example east-west timeseries example

[as1234554321]

haii @oguzhannysr hello, how do I get this much decompose?, I only got 1 UD file can you help me to tells how to process decomposition appropriately and slowly. Thank You

[yumorishita]

How technically correct is it to separate each image pair as east-west and up-down using the cumulative los data produced for the ascending and descending orbits as of 2016?

What do you mean by "as of 2016"?

[oguzhannysr]

2016 yılı itibariyle yükselen ve alçalan yörüngeler için üretilen kümülatif los verilerini kullanarak her görüntü çiftini doğu-batı ve yukarı-aşağı olarak ayırmak teknik olarak ne kadar doğrudur?

"2016 itibariyle" derken neyi kastediyorsunuz?

@yumorishita, I meant to say starting from 2016 and until 2024, so I did my work within this date range.

[yumorishita]

How did you treat the date difference between ascending and descending in the decomposition?

[oguzhannysr]

@yumorishita , as it can be seen in the image, after the date 20211218, the dates of the couples start not to match with each other. Therefore, I made ud-ew separation based on two pairs in 6-day periods. For example, I matched the date 20211224 in the ascending orbit with the date 20211230 in the descending orbit and made the separation as ud-ew. How technically correct is this? Does it provide me with the correct trend of the deformation?

[yumorishita]

If the deformation is slow and we can ignore the 6 day difference, I think your approach makes sense.

[oguzhannysr]

@yumorishita , Thank you sir, but is the reason why the data dates do not match after a certain time (after 2021) due to the failure of the Sentinel-1 satellite or due to incomplete processing of data in the COMET system?

[yumorishita]

I don't know. You should check it yourself by searching for data, e.g., on ASF.

[oguzhannysr]

dear @yumorishita , Is there an anomaly in n_loop_err in my results here? Do you think I should set p15_n_loop_err_thre=0 and run it again? 202409120903batch_LiCSBAS_01_16.log

Noise index    : Threshold  (rate to be masked)
- coh_avg      : 0.05       ( 0.0%)
- n_unw        :  531       ( 0.0%)
- vstd         :  100 mm/yr ( 0.0%)

- maxTlen      :    1 yr    ( 0.0%)
- n_gap        :   10       ( 0.0%)
- stc          :    5 mm    ( 0.0%)
- n_ifg_noloop :   50       ( 0.5%)
- n_loop_err   :    5       (20.2%)
- resid_rms    :    2 mm    ( 2.5%)

Masked pixels  : 224/1109 (20.2%)
Kept pixels    : 885/1109 (79.8%)

[yumorishita]

The result looks normal.

[oguzhannysr]

@yumorishita , I got an error like the following. I couldn't understand why. My other files are below. What do you think is the reason?

202409190909batch_LiCSBAS_01_16.log batch_LiCSBAS.txt

Create unw and cc
  1 parallel processing...
    0/1258th IFG...
/home/yasar/LICSBAS/bin/LiCSBAS02_ml_prep.py:441: RuntimeWarning: invalid value encountered in cast
  cc = cc.astype(np.uint8) ##nan->0, max255, auto-floored
/home/yasar/LICSBAS/bin/LiCSBAS02_ml_prep.py:441: RuntimeWarning: invalid value encountered in cast
  cc = cc.astype(np.uint8) ##nan->0, max255, auto-floored
/home/yasar/LICSBAS/bin/LiCSBAS02_ml_prep.py:441: RuntimeWarning: invalid value encountered in cast
  cc = cc.astype(np.uint8) ##nan->0, max255, auto-floored

[as1234554321]

@yumorishita I got some error, please help me, thankyou 1) the mask pixel percentage is very high is that normal? 2)lots of ifg nan Does it have to be deleted manually or is it deleted automatically? 3)What does this image mean and what does it affect? /home/winda2/miniconda3/envs/licsbas/lib/python3.12/site-packages/numpy/lib/nanfunctions.py:1879: RuntimeWarning: Degrees of freedom <= 0 for slice. var = nanvar(a, axis=axis, dtype=dtype, out=out, ddof=ddof,

[yumorishita]

@oguzhannysr I am not sure but cc files may be corrupted. Please check the contents.

[yumorishita]

@as1234554321

1. It can be normal
2. automatically deleted
3. I have no idea without the full log

[yumorishita]

These messages are just warnings, not errors, and can be ignored. The processing was successfully finished with no error.

[oguzhannysr]

@yumorishita , Now I'm stuck here, which n_para should I change?

  Running  64000/ 65117th point...
  Running  63000/ 65117th point...
  Running  61000/ 65117th point...
  Running  62000/ 65117th point...
  Running  65000/ 65117th point...
  Running  60000/ 65117th poi
```nt...
  Elapsed time for 11th patch: 00h 05m 58s

Process 624/2689th line (12/52th patch)...
  Reading 724 ifg's unw data...
  84278/183768 points removed due to not enough ifg data...

  Identifing gaps, and counting n_gap and n_ifg_noloop,
  with 16 parallel processing...
    Running  1/16th patch...
    Running  2/16th patch...
    Running  3/16th patch...
    Running  4/16th patch...
    Running  5/16th patch...
    Running  6/16th patch...
    Running  9/16th patch...
    Running 10/16th patch...
    Running 11/16th patch...
    Running 12/16th patch...
    Running 14/16th patch...
    Running 15/16th patch...
    Running 16/16th patch...
    Running  8/16th patch...
    Running  7/16th patch...
    Running 13/16th patch...

@yumorishita ,Hello sir, the problem still persists. I want to process the entire tile at once, but I stay in the same step. I can't understand whether my computer's RAM is not enough.

[yumorishita]

@oguzhannysr What is your RAM size?

[yumorishita]

@as1234554321 I have no idea from this limited information.

[oguzhannysr]

@oguzhannysrRAM boyutunuz nedir?

@yumorishita , 16 gb

[as1234554321]

@as1234554321 I have no idea from this limited information.

I have processed the land subsidence using licsbas batch by changing the "y" parameter in the gacos, mask and clip steps, others are default. than, I decompose result of ascending and desending However, the results are very different from the journal which states that all of these areas experienced land subsidence, while the results of Licsbas in the southern part experienced uplift. Below I attach the difference between the journal and the LICSBAS results. What causes this anomaly? and if it is due to licsbas processing, which part should be changed? Thank You 202409191148batch_LiCSBAS_01_16.log 202409191744batch_LiCSBAS_01_16.log

[as1234554321]

sorry sir, I ask different question I have stuck proses in this step, What should I do? 202409221809batch_LiCSBAS_01_16.log

[yumorishita]

@oguzhannysr 16GB should be generally enough. Is your environment WSL? Which OS do you use?

[yumorishita]

@as1234554321 I think LiCSBAS also shows subsidence and they are consistent. Regarding stuck, see https://github.com/yumorishita/LiCSBAS/issues/244#issuecomment-1500877478.

[as1234554321]

@as1234554321Saya pikir LiCSBAS juga menunjukkan penurunan dan hasilnya konsisten. Mengenai macet, lihat #244 (komentar) .

but in south area uplift, I think should subsidence but the result uplift.what cause this different?

[oguzhannysr]

@oguzhannysr 16GB should be generally enough. Is your environment WSL? Which OS do you use?

@yumorishita, Yes, I run LICSBAS in WSL in Windows.

[yumorishita]

@as1234554321 Now I see your point. You should set the reference point in the south area.

[yumorishita]

@oguzhannysr I have never used WSL but it seems that the stuck tends to happen in WSL according to the reported issues. I suggest using the VirtualBox as I do.

[as1234554321]

hi sir @yumorishita i was set reference point with -- licsbas cum2vel --ref.geo but the result same. the velocity of the south area remains positive/uplift. pic 1 ( process licsbas cum2vel done) pic 2 the plot LiCSBAS_plot_ts.py of the result licsbascum2 vel. left is ascending and right is desending. both of them in south area, velocity value still positif, but in other journal/paper this area was landsubsidence. what causes this difference?

[yumorishita]

step05 does not set any reference point but just clips the area. The auto-determined reference is selected using the median under the assumption that more than half of the pixels are stable. In your case, this assumption does not hold and the stable area falsely has uplift. The option --ref_geo 110.22/110/59/-7.19/-6.90 specifies the entire area, not the southern area. You must specify the small area in the stable southern area, e.g., -r 200:210/150:160.

[as1234554321]

Ohh I see, thankyou for your answer and suggestion 1.ref point stable means an area where there is no uplift/land subsidence, is that right/wrong? 2.how to determine a stable ref point?

3. i was ran your suggestion to change ref area -r 200:210/150:160 with licsbascum2vel.py in ascending it works, but when displaying the results with licsbasplot ref the area at the top left of the velocity window does not change. here's the picture. did I take the wrong steps?

4. I was ran point no 3 in decending but the result nan value. I was confused about this. Another journal explains that the southern area has land subsidence but the result is licsbas uplift. sorry for many quetions in this part, I hope you don't mind answering it. Thankyou

[yumorishita]

1. Right.
2. It needs external information or can be arbitrary.
3. You should also set -r option for LiCSBAS_plot_ts.py
4. I guess there was no valid point in the selected reference area. You should change the reference area.

[as1234554321]

1. Benar.
2. Memerlukan informasi eksternal atau dapat bersifat acak -Can the GPS CORS point be used as a ref point? If it is determined randomly, the result is likely to be a nan value like what I did before. -can you suggestion to me for ref point are correct?
3. Anda juga harus mengatur -ropsi untuk LiCSBAS_plot_ts.py Can you give a correct example of how to write it so that it is successful?
4. Saya kira tidak ada titik yang valid di area referensi yang dipilih. Anda harus mengubah area referensi. is there an example where changing the ref point will change the velocity value from (+) to (-)?

[yumorishita]

The pixel corresponding to the GPS CORS can be a ref point if it exists. Please read the usage of LiCSBAS_plot_ts.py. Your case is the good example if you change the ref point.

[as1234554321]

thankyou sir, I will try it permission to make sure sir, so 1.I changed the reference point using licsbascum2vel.py then run on licsbasplot.py + licsbascum2vel result + -r GPS CORS point. is that right?

2. Hi sir, I want to change the ref point using gps cors. but the available data is in lat/lot. How to change it to pixel because -r (X1/Y1....). here are the coordinates. can you help me to give an example Lat/lot GPS CORS : -6. 987259/110.37693

   Thank you

[yumorishita]

licsbascum2vel.py does not affect licsbasplot.py at all because the output of licsbascum2vel.py is not used in licsbasplot.py. You can use --ref_geo option instead of -r. Please read the usage.

[as1234554321]

licsbascum2vel.py sama sekali tidak memengaruhi licsbasplot.py karena output licsbascum2vel.py tidak digunakan dalam licsbasplot.py. Anda dapat menggunakan opsi --ref_geo sebagai ganti -r. Harap baca penggunaannya.

so what tools do i use to change the ref point and how do i know if the velocity has changed? sorry I don't know how to do it, thankyou

[yumorishita]

You should specify the reference for each command.

[as1234554321]

Anda harus menentukan referensi untuk setiap perintah.

yes I know I have to change the ref on every command. but I'm confused about is what commands should be used? Can you provide step by step tools that should be used. I've read the issue related to the ref area but I still miss understanding. thankyou

102 #131