search

acolite / acolite

ACOLITE: generic atmospheric correction module
GNU General Public License v3.0
158 stars 62 forks source link

run acolite tact, but it is killed #37

Closed bluesteven closed 2 years ago

bluesteven commented 2 years ago

Running ACOLITE processing - Generic GitHub Clone Run ID - 20220731_171156 Starting conversion of 1 scenes Starting conversion of /home/heh/acolite/landsat/LC08_L1TP_119043_20210114_20210308_01_T1 Importing metadata from /home/heh/acolite/landsat/LC08_L1TP_119043_20210114_20210308_01_T1 Writing geolocation lon/lat Wrote lon Wrote lat Converting bands Converting bands: Wrote rhot_443 ((7721, 7571)) Converting bands: Wrote rhot_483 ((7721, 7571)) Converting bands: Wrote rhot_561 ((7721, 7571)) Converting bands: Wrote rhot_655 ((7721, 7571)) Converting bands: Wrote rhot_865 ((7721, 7571)) Converting bands: Wrote rhot_1609 ((7721, 7571)) Converting bands: Wrote rhot_2201 ((7721, 7571)) Converting bands: Wrote rhot_592 to separate L1R_pan Converting bands: Wrote rhot_592 ((7721, 7571)) Converting bands: Wrote rhot_1373 ((7721, 7571)) Converting bands: Wrote bt10 Converting bands: Wrote bt11 Conversion took 39.5 seconds Created /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc Wrote /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R_rgb_rhot.png Running acolite for /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc Using DSF atmospheric correction Processing with 1482 tiles (39x38 tiles of 200x200 pixels) Writing lon Wrote lon Writing lat Wrote lat sza not found in /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc vza not found in /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc raa not found in /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc Loading LUTs romix+rsky_t True Loading LUTs took 1.1 s 1 rhot_443 L8_OLI/B1 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B1 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 2 rhot_483 L8_OLI/B2 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B2 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 3 rhot_561 L8_OLI/B3 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B3 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 4 rhot_655 L8_OLI/B4 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B4 ACOLITE-LUT-202110-MOD2 took 0.003s (StdLUT) 5 rhot_865 L8_OLI/B5 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B5 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 6 rhot_1609 L8_OLI/B6 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B6 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 7 rhot_2201 L8_OLI/B7 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B7 ACOLITE-LUT-202110-MOD2 took 0.004s (StdLUT) 8 rhot_592 L8_OLI/B8 ACOLITE-LUT-202110-MOD1 took 0.004s (StdLUT) L8_OLI/B8 ACOLITE-LUT-202110-MOD2 took 0.003s (StdLUT) Choosing best fitting model: min_drmsd (2 bands) Computing RMSD for model ACOLITE-LUT-202110-MOD1 Computing RMSD for model ACOLITE-LUT-202110-MOD2 Selecting most common model for processing. ACOLITE-LUT-202110-MOD1: 5.5%: mean aot of subset = 0.29 ACOLITE-LUT-202110-MOD2: 94.5%: mean aot of subset = 0.19 Selected ACOLITE-LUT-202110-MOD2, mean aot = 0.19 use_revlut False Wrote rhot_443 Computing surface reflectance 1 443 0.998 Interpolating tiles Wrote rhos_443 L8_OLI/B1 took 7.3s (StdLUT) Wrote rhot_483 Computing surface reflectance 2 483 0.987 Interpolating tiles Wrote rhos_483 L8_OLI/B2 took 6.7s (StdLUT) Wrote rhot_561 Computing surface reflectance 3 561 0.930 Interpolating tiles Wrote rhos_561 L8_OLI/B3 took 6.8s (StdLUT) Wrote rhot_655 Computing surface reflectance 4 655 0.953 Interpolating tiles Wrote rhos_655 L8_OLI/B4 took 6.7s (StdLUT) Wrote rhot_865 Computing surface reflectance 5 865 0.997 Interpolating tiles Wrote rhos_865 L8_OLI/B5 took 6.6s (StdLUT) Wrote rhot_1609 Computing surface reflectance 6 1609 0.959 Interpolating tiles Wrote rhos_1609 L8_OLI/B6 took 6.6s (StdLUT) Wrote rhot_2201 Computing surface reflectance 7 2201 0.919 Interpolating tiles Wrote rhos_2201 L8_OLI/B7 took 6.8s (StdLUT) Wrote rhot_592 Computing surface reflectance 8 592 0.938 Interpolating tiles Wrote rhos_592 L8_OLI/B8 took 7.1s (StdLUT) Wrote rhot_1373 Computing orange band Wrote rhos_613 Wrote /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L2R.nc Wrote /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L2R_rgb_rhos.png Running tact for /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc Killed

bluesteven commented 2 years ago

libradtran is installed and the last command "make check" is all right

acolite commented 2 years ago

Hi Steven

You may have run out of RAM, there was an unnecessarily quite memory intensive step at the beginning of the TACT run, especially so for a full scene. The last commit should probably fix this: https://github.com/acolite/acolite/commit/187e1210a1632a38b91462f272e2c7c370c15c79

Let me know if it worked! If not, did any files get produced in data/TACT/grid/?

Quinten

bluesteven commented 2 years ago

Thanks! It works. But I get another problem: rote /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R_rgb_rhot.png Running tact for /home/heh/acolite/output/L8_OLI_2021_01_14_02_33_21_119043_L1R.nc Wrote lon to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote lat to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote tau10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote Lu10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote Ld10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote tau11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote Lu11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote Ld11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote bt10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote lt10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote ls10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote em10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote st10 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote bt11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote lt11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote ls11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote em11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote st11 to /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST.nc Settings file L8_TIRS not found. Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_tau10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_Lu10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_Ld10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_tau11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_Lu11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_Ld11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_bt10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_lt10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_ls10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_em10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_st10.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_bt11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_lt11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_ls11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_em11.png Wrote /home/heh/acolite/output/L8_TIRS_2021_01_14_02_33_21_119043_ST_st11.png

what is the "Settings file L8_TIRS" ?

bluesteven commented 2 years ago

My setting file is following:

ACOLITE settings

Written at 2022-07-31 09:29:50

output=/home/heh/acolite/output output_bt=False atmospheric_correction=False pans_rgb_rhot=False pans_rgb_rhos=False pans_export_geotiff_rgb=False l1r_export_geotiff=False l2t_export_geotiff=False l2r_export_geotiff=False l2w_export_geotiff=False l1r_export_geotiff_rgb=False l2r_export_geotiff_rgb=False l1r_delete_netcdf=False l2t_delete_netcdf=False l2r_delete_netcdf=False l2r_pans_delete_netcdf=False l2w_delete_netcdf=False delete_acolite_run_text_files=False delete_extracted_input=False inputfile=/home/heh/acolite/landsat/LC08_L1TP_119043_20210114_20210308_01_T1 tact_run=True tact_map=True tact_output_atmosphere=True tact_output_intermediate=True

bluesteven commented 2 years ago

I am not sure if the settings file is wrong.

bluesteven commented 2 years ago

Maybe my computer's memory is too low（win11+wsl2，16GB）, it is still killed sometimes.

acolite commented 2 years ago

Settings file looks fine! Good that you disable atmospheric correction / OLI processing if you don't need it. Most of the other options you have default to False, so you don't really need them in your settings file.

I have not used Win11 and WSL2 myself, but how much memory is allocated to WSL2? It seems the default is 1/2 of the available RAM. Do you have the option to run for a region of interest? (e.g. using a S,W,N,E limit box, or by supplying a polygon file)

Quinten

bluesteven commented 2 years ago

"Settings file L8_TIRS not found" still present sometimes. No more get killed after limit box is used. Thanks!

acolite commented 2 years ago

Thanks for the update! It seems like it was a RAM issue when processing full scenes under WSL. Will close the issue.

The settings file message is no problem! There are no specific settings to TIRS yet, so I did not add those defaults files. I have now added empty ones so the error should disappear: https://github.com/acolite/acolite/commit/fa3c7399e0f9f7f7f31ee092e8cb5edff1078fc8

Quinten