search

yumorishita / LiCSBAS2

GNU General Public License v3.0
26 stars 4 forks source link

Error in LiCSBAS02_ml_prep.py with color #19

Closed ClementGold closed 1 year ago

ClementGold commented 1 year ago

Hello,

I use the batch file.sh and I get this error :

LiCSBAS02_ml_prep.py ver1.7.4 20201119 Y. Morishita LiCSBAS02_ml_prep.py -i GEOC -n 1 Traceback (most recent call last): File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS02_ml_prep.py", line 452, in sys.exit(main()) ^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS02_ml_prep.py", line 128, in main cmap_wrap = tools_lib.get_cmap('cm_insar') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/LiCSBAS_lib/LiCSBAS_tools_lib.py", line 417, in get_cmap _cmap = cm_insar() ^^^^^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/LiCSBAS_lib/LiCSBAS_tools_lib.py", line 139, in cm_insar return LSC.from_list('cm_insar', rgbs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/clement.goldmann/opt/anaconda3/envs/LiCSBAS/lib/python3.11/site-packages/matplotlib/colors.py", line 1056, in from_list r, g, b, a = to_rgba_array(colors).T ^^^^^^^^^^^^^^^^^^^^^ File "/Users/clement.goldmann/opt/anaconda3/envs/LiCSBAS/lib/python3.11/site-packages/matplotlib/colors.py", line 454, in to_rgba_array raise ValueError("RGBA values should be within 0-1 range") ValueError: RGBA values should be within 0-1 range

I precise this step works fine with the LiCSBAS original code, I try now with LiCSBAS2 because I got an error on the step 11.

Best regards, Clément

ClementGold commented 1 year ago

I'm just precise, I used Python 3.11 on a Mac OS Ventura

file "/Users/clement.goldmann/opt/anaconda3/envs/LiCSBAS/lib/python3.11/site-packages/matplotlib/colors.py", line 1056, in from_list r, g, b, a = to_rgba_array(colors).T ^^^^^^^^^^^^^^^^^^^^^ File "/Users/clement.goldmann/opt/anaconda3/envs/LiCSBAS/lib/python3.11/site-packages/matplotlib/colors.py", line 454, in to_rgba_array

yumorishita commented 1 year ago

I cannot reproduce the error with python3.11.3. Could you please show me the STDOUT of LiCSBAS_check_install.py? My environment is:

  astropy(5.1) OK
  bs4(4.12.2) OK
  h5py(3.7.0) OK
  matplotlib(3.7.1) OK
  numpy(1.24.3) OK
  psutil(5.9.0) OK
  requests(2.29.0) OK
  statsmodels(0.13.5) OK
  gdal(3.6.2) OK
ClementGold commented 1 year ago

Yes here is it :

Python version: 3.11.3 OK

Check required modues and versions astropy(5.2.2) OK bs4(4.12.2) OK h5py(3.8.0) OK matplotlib(3.7.1) OK numpy(1.24.3) OK psutil(5.9.5) OK requests(2.29.0) OK statsmodels(0.13.5) OK gdal(3.6.4) OK

idk if it can be influence but i used the LiCSBAS 1 first and i got an error step 11 so as you mentionned to used the LiCSBAS2 to solve the error I did all installation in the folder LiCSBAS2 and did all the step

Capture d’écran 2023-05-03 à 14 17 40
yumorishita commented 1 year ago

Can you try inserting rgbs[rgbs>1] = 1 to https://github.com/yumorishita/LiCSBAS2/blob/9c76a2fbefea03b39c1894460480439632b30877/LiCSBAS_lib/LiCSBAS_tools_lib.py#L138 and running again?

ClementGold commented 1 year ago

Seems working until process 4

LiCSBAS04op_mask_unw.py ver1.3.5 20210105 Y. Morishita LiCSBAS04op_mask_unw.py -i GEOCml1GACOS -o GEOCml1GACOSmask

ERROR: Neither -r nor -f option is given!

For help, use -h or --help.

I will check the process 4 in my batch command

yumorishita commented 1 year ago

Thanks. I have updated the code. Regarding the error in step04, you must set at least one of the following three parameters: https://github.com/yumorishita/LiCSBAS2/blob/de5ca0ffab2e7aa76f5c7f8dbfda7de74bf17166/batch_LiCSBAS.sh#L36-L38

ClementGold commented 1 year ago

I just set 0.2 of the coh thre but I get this error :

FutureWarning: In the future np.bool will be defined as the corresponding NumPy scalar. bool_mask = np.zeros((length, width), dtype=np.bool) Traceback (most recent call last): File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS04op_mask_unw.py", line 325, in sys.exit(main()) ^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS04op_mask_unw.py", line 175, in main bool_mask = np.zeros((length, width), dtype=np.bool) ^^^^^^^ File "/Users/clement.goldmann/opt/anaconda3/envs/LiCSBAS/lib/python3.11/site-packages/numpy/init.py", line 305, in getattr raise AttributeError(__former_attrs_[attr]) AttributeError: module 'numpy' has no attribute 'bool'. np.bool was a deprecated alias for the builtin bool. To avoid this error in existing code, use bool by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'bool_'?

yumorishita commented 1 year ago

Thank you for reporting the error. I have fixed the bug. Please update your local LiCSBAS2 codes and try again.

ClementGold commented 1 year ago

That work now,

LiCSBAS11_check_unw.py -d GEOCml1GACOSmaskclip

coh_thre : 0.05 unw_cov_thre : 0.3

Size : 2861 x 2791

Reading unw and cc data... 0/ 76th unw to identify valid area... 0/ 76th cc and unw... /Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS11_check_unw.py:232: RuntimeWarning: Mean of empty slice coh_avg_ifg.append(np.nanmean(coh[bool_valid])) # Use valid area only /Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS11_check_unw.py:234: RuntimeWarning: invalid value encountered in divide rate_cov = np.array(n_unw_ifg)/n_unw_valid

76/76 ifgs are discarded from further processing. ifg dates unw_cov coh_av 20210108_20210120 nan nan 20210108_20210225 nan nan 20210120_20210201 nan nan 20210120_20210321 nan nan 20210201_20210213 nan nan 20210201_20210321 nan nan 20210213_20210225 nan nan 20210213_20210321 nan nan 20210213_20210402 nan nan 20210213_20210414 nan nan 20210225_20210321 nan nan 20210225_20210402 nan nan 20210225_20210414 nan nan 20210225_20210426 nan nan 20210321_20210402 nan nan 20210321_20210414 nan nan 20210321_20210426 nan nan 20210321_20210508 nan nan 20210321_20210613 nan nan 20210321_20210625 nan nan 20210402_20210414 nan nan 20210402_20210426 nan nan 20210402_20210508 nan nan 20210402_20210520 nan nan 20210414_20210426 nan nan 20210414_20210508 nan nan 20210414_20210520 nan nan 20210414_20210613 nan nan 20210426_20210508 nan nan 20210426_20210520 nan nan 20210426_20210613 nan nan 20210426_20210625 nan nan 20210508_20210520 nan nan 20210508_20210613 nan nan 20210508_20210625 nan nan 20210508_20210707 nan nan 20210520_20210613 nan nan 20210520_20210625 nan nan 20210520_20210707 nan nan 20210520_20210719 nan nan 20210613_20210625 nan nan 20210613_20210707 nan nan 20210613_20210719 nan nan 20210625_20210707 nan nan 20210625_20210719 nan nan 20210625_20210731 nan nan 20210625_20210812 nan nan 20210707_20210719 nan nan 20210707_20210731 nan nan 20210707_20210812 nan nan 20210707_20210824 nan nan 20210719_20210731 nan nan 20210719_20210812 nan nan 20210719_20210824 nan nan 20210719_20210917 nan nan 20210731_20210812 nan nan 20210731_20210824 nan nan 20210731_20210917 nan nan 20210731_20210929 nan nan 20210812_20210824 nan nan 20210812_20210917 nan nan 20210812_20210929 nan nan 20210824_20210917 nan nan 20210824_20210929 nan nan 20210917_20210929 nan nan 20210917_20211023 nan nan 20210929_20211023 nan nan 20210929_20211104 nan nan 20211023_20211104 nan nan 20211023_20211128 nan nan 20211104_20211128 nan nan 20211104_20211210 nan nan 20211104_20211222 nan nan 20211128_20211210 nan nan 20211128_20211222 nan nan 20211210_20211222 nan nan Traceback (most recent call last): File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS11_check_unw.py", line 356, in sys.exit(main()) ^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS11_check_unw.py", line 311, in main raise ValueError('All ifgs are regarded as bad!\nChange the parameters or check the input ifgs.\n') ValueError: All ifgs are regarded as bad! Change the parameters or check the input ifgs.

yumorishita commented 1 year ago

Should I investigate the above error? It is because no valid pixels are included in the clipped area.

ClementGold commented 1 year ago

This error indicate a pixel value are not good ? I took 1 year of Jakarta data which normally have enough data, maybe i can change the coherence treshold ?

ClementGold commented 1 year ago

Changing the coherence treshold doesn't change the result unfortunately

yumorishita commented 1 year ago

I guess either step03 GACOS, step04 mask, or step05 clip removed all valid pixels. Please check GEOCml1*/*/*.png and identify which step was wrong.

ClementGold commented 1 year ago

I guess either step03 GACOS, step04 mask, or step05 clip removed all valid pixels. Please check GEOCml1*/*/*.png and identify which step was wrong.

It's the step 04 which doesn't work because on GEOCml1GACOSmask I have no data like : 20210108_20210120 unw

Here is Gacos PNG 20210120_20210201 gacos

ClementGold commented 1 year ago

I have removed the part 4 and 5, step 11 work but from 12 that bug :

1st Loop closure check and make png for all possible 94 loops, with 8 parallel processing... 0/ 94th loop...

2nd Loop closure check without bad ifgs to define ref area... with 8 parallel processing... 0/ 94th loop... /Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS12_loop_closure.py:386: RuntimeWarning: All-NaN slice encountered refyx = np.where(loop_ph_rms_points_masked==np.nanmin(loop_ph_rms_points_masked)) Traceback (most recent call last): File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS12_loop_closure.py", line 880, in sys.exit(main()) ^^^^^^ File "/Users/clement.goldmann/LiCSBAS/LiCSBAS2/LiCSBAS2/bin/LiCSBAS12_loop_closure.py", line 387, in main refy1 = refyx[0][0] # start from 0, not 1


IndexError: index 0 is out of bounds for axis 0 with size 0

It's maybe a dumb question but to clean all the stuff it's better to remove for example the folder GEOCml1GACOSmask ?
ClementGold commented 1 year ago

I try on different region, which was Italia Venezia working good and do it again in Indonesia Jakarta area but I needed to clip the area in question

yumorishita commented 1 year ago

The error in step12 is because all pixels are affected by unwrapping errors.

ClementGold commented 1 year ago

Seems coming to a precise area because when I clip the error disapears

ditafaith commented 1 year ago

Seems coming to a precise area because when I clip the error disapears

I tried to work with LiCSBAS (old version) in same area with yours, and it works.

Hi Morishita, may I know what are the differences between LiCSBAS2 ? Should I change to the newer version?

yumorishita commented 1 year ago

All the changes from LiCSBAS can be found https://github.com/yumorishita/LiCSBAS2/pulls?q=is%3Apr+is%3Aclosed or https://github.com/yumorishita/LiCSBAS2/commits/master. You may use LiCSBAS as long as it works because there are no significant functional changes from LiCSBAS.

ditafaith commented 1 year ago

All the changes from LiCSBAS can be found https://github.com/yumorishita/LiCSBAS2/pulls?q=is%3Apr+is%3Aclosed or https://github.com/yumorishita/LiCSBAS2/commits/master. You may use LiCSBAS as long as it works because there are no significant functional changes from LiCSBAS.

Thanks. I still use the old one, since there is no issue.