search

GeoscienceAustralia / PyRate

A Python tool for estimating velocity and time-series from Interferometric Synthetic Aperture Radar (InSAR) data.
https://geoscienceaustralia.github.io/PyRate/
Apache License 2.0
200 stars 70 forks source link

Memory error #308

Closed EllenLee183 closed 3 years ago

EllenLee183 commented 3 years ago

When I want to try to correct orbit errors by using network orbital correction, it encounters to a problem ,"memory error: unable to allocate 350. gib for an arrary with shape(70026800,134) and data type float32"

mcgarth commented 3 years ago

Hi @EllenLee183 thanks for the question! It sounds like the problem is too big for the RAM you have available to you on your machine. Have you set additional multi-looking via the orbfitlksx and orbfitlksy parameters? If no I would suggest doing this and if yes you might try increasing the values (say from 10 to 20). Since orbital errors are usually long wavelength you a fairly safe to do the correction on a highly-subsampled version of the interferograms.

Hope this helps, but let me know if not!

EllenLee183 commented 3 years ago

I am very pleased to receive your letter. It is true that my data volume is too large, which causes the relative available memory on the machine to be too small. Later I debugged orbfitlksx and orbfitlksy parameters and it was able to run successfully! But I have a small suggestion, whether it is possible to add block processing function to improve calculation efficiency.

Many regards, Ellen Lee

------------------ 原始邮件 ------------------ 发件人: "GeoscienceAustralia/PyRate" <notifications@github.com>; 发送时间: 2020年11月16日(星期一) 中午1:20 收件人: "GeoscienceAustralia/PyRate"<PyRate@noreply.github.com>; 抄送: "逆水行舟"<2722320749@qq.com>;"Mention"<mention@noreply.github.com>; 主题: Re: [GeoscienceAustralia/PyRate] Memory error (#308)

Hi @EllenLee183 thanks for the question! It sounds like the problem is too big for the RAM you have available to you on your machine. Have you set additional multi-looking via the orbfitlksx and orbfitlksy parameters? If no I would suggest doing this and if yes you might try increasing the values (say from 10 to 20). Since orbital errors are usually long wavelength you a fairly safe to do the correction on a highly-subsampled version of the interferograms.

Hope this helps, but let me know if not!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

mcgarth commented 3 years ago

Thanks for the suggestion @EllenLee183 and I am glad you were able to run your processing. We have an open issue here https://github.com/GeoscienceAustralia/PyRate/issues/75 that I think would address your idea. Feel free to add a comment in there.

EllenLee183 commented 3 years ago

Thanks again for your reply. good luck

------------------ 原始邮件 ------------------ 发件人: "Matt Garthwaite"<notifications@github.com>; 发送时间: 2020年11月17日(星期二) 上午6:05 收件人: "GeoscienceAustralia/PyRate"<PyRate@noreply.github.com>; 抄送: "逆水行舟"<2722320749@qq.com>; "Mention"<mention@noreply.github.com>; 主题: Re: [GeoscienceAustralia/PyRate] Memory error (#308)

Thanks for the suggestion @EllenLee183 and I am glad you were able to run your processing. We have an open issue here #75 that I think would address your idea. Feel free to add a comment in there.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.