yumorishita / LiCSBAS2

GNU General Public License v3.0
28 stars 5 forks source link

When running the LiCSBAS16_filt_ts.py script, the log shows the following message: "Out of memory: Kill process." #82

Open xuliangye opened 1 month ago

xuliangye commented 1 month ago
  When running the LiCSBAS16_filt_ts.py script, the terminal automatically closes, and the log shows the following message: "Out of memory: Kill process."
 I have searched online for many solutions, but none have resolved the issue. The computer has enough memory; could it be a problem with the program? Perhaps there are too many iterations? I apologize for the intrusion.
yumorishita commented 1 month ago

It means the computer does not have enough memory. Try

xuliangye commented 1 month ago

It means the computer does not have enough memory. Try

  • increase nlook in step02
  • clip small area in step05
  • use small filter width in step16
  Thank you very much, Mr. Yu. The images I used for calculation were not cropped, and the size is one frame (as required for the research). 
 The code running in step 16 is: LiCSBAS16_filt_ts.py -t TS_GEOCml1clip, with no parameters. How can I specifically implement what Mr. Yu mentioned about "using a small filter width in step 16"? Thank you very much!
yumorishita commented 1 month ago

https://github.com/yumorishita/LiCSBAS2/blob/f1e2592dfb37cfde243f644fcfe554ab4b67bba7/batch_LiCSBAS.sh#L61-L62

xuliangye commented 1 month ago

    Sorry for replying to you so late, Mr. Yu. I have another question to consult you. I changed to another computer with a graphics card and 128 GB of system memory to process an image (without cropping), but the process was also terminated halfway when running on the same terminal.

    The data I am processing includes 431 images, each of its original size (not cropped, 250 km × 250 km). I ran the command: LiCSBAS16_filt_ts.py -t TS_GEOCml1clip, without modifying any parameters.

     May I ask if this program can only handle relatively small areas? If using this program, what is the general required computer memory for processing large areas (such as a image not cropped), or what kind of processing methods are needed?       Thank you very much, Mr. Yu.  I sincerely look forward to your reply!  

1124433626 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Yu @.>; 发送时间: 2024年10月24日(星期四) 晚上8:11 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [yumorishita/LiCSBAS2] When running the LiCSBAS16_filt_ts.py script, the log shows the following message: "Out of memory: Kill process." (Issue #82)

https://github.com/yumorishita/LiCSBAS2/blob/f1e2592dfb37cfde243f644fcfe554ab4b67bba7/batch_LiCSBAS.sh#L61-L62

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

yumorishita commented 1 month ago

I have never tried such a very large dataset and have no idea about the general required computer memory.

xuliangye commented 1 month ago

  I really appreciate your letter and your patient explanations, Mr. Yu. I'll keep looking for other solutions!  

1124433626 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Yu @.>; 发送时间: 2024年10月27日(星期天) 晚上7:49 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [yumorishita/LiCSBAS2] When running the LiCSBAS16_filt_ts.py script, the log shows the following message: "Out of memory: Kill process." (Issue #82)

I have never tried such a very large dataset and have no idea about the general required computer memory.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

xuliangye commented 4 weeks ago

Dear Professor:

    I apologize for bothering you again. I have read your paper [1],would like to consult you on a few points:

    1)In this paper, when processing large areas (the entire Japan), how do you ensure that different target areas have overlapping regions during the cropping process? Does the LICSABS software provide existing code for this step?

   2)After processing different target areas separately, how do you merge these results? In your paper, you mentioned selecting stable points manually as common reference points and then using least squares for estimation. How is this implemented? Is it done using the LICSABS software, or is there another method?     Thank you very much, and I sincerely look forward to your reply!

References:

[1]Morishita, Y. (2021). "Nationwide urban ground deformation monitoring in Japan using Sentinel-1 LiCSAR products and LiCSBAS." Progress in Earth and Planetary Science 8(1).

1124433626 @.***

 

------------------ 原始邮件 ------------------ 发件人: "yumorishita/LiCSBAS2" @.>; 发送时间: 2024年10月27日(星期天) 晚上7:49 @.>; @.**@.>; 主题: Re: [yumorishita/LiCSBAS2] When running the LiCSBAS16_filt_ts.py script, the log shows the following message: "Out of memory: Kill process." (Issue #82)

I have never tried such a very large dataset and have no idea about the general required computer memory.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

yumorishita commented 4 weeks ago
  1. Sorry but I cannot understand your question.
  2. Which part are you mentioning? I did not merge any results.
xuliangye commented 4 weeks ago

   In reference [1] 2.3 Methods(Page 4 of 23-Page 5 of 23):

   “ In this study, a common reference point for an area among different frames was further set to compare or combine the results from multiple frames by computing the average RMS among all available frames for an area and selecting a pixel with the minimum averaged RMS. Locations for a stable GNSS station outside of the significant deformation area were manually selected as the common reference point in Sanjo (ID: 153) and Kumamoto (ID: 431), because most of the processed area was deformed, and the automatically selected point was not stable.”

   After manually selecting the reference points in the above section, how do you calculate and obtain the deformation of the study area? Do you use the LICSABS software or another method? Can you provide specific details on how this is done.

References:

[1] Morishita, Y. (2021). "Nationwide urban ground deformation monitoring in Japan using Sentinel-1 LiCSAR products and LiCSBAS." Progress in Earth and Planetary Science 8(1).

 

1124433626 @.***

 

------------------ 原始邮件 ------------------ 发件人: "Yu @.>; 发送时间: 2024年11月1日(星期五) 晚上9:39 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [yumorishita/LiCSBAS2] When running the LiCSBAS16_filt_ts.py script, the log shows the following message: "Out of memory: Kill process." (Issue #82)

Sorry but I cannot understand your question.

Which part are you mentioning? I did not merge any results.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

yumorishita commented 3 weeks ago

I used LiCSBAS_cum2tstxt.py and LiCSBAS_cum2vel.py with the --ref_geo option.