search

ARCCSS-extremes / climpact

Calculate the ET-SCI climate extremes indices
GNU General Public License v3.0
42 stars 29 forks source link

How to make the calculation faster? #92

Open JinZarrin opened 2 years ago

JinZarrin commented 2 years ago

Hi there, I'm now running the Climpact2 package through R to calculate indices for the gridded nc.file, but I found it took more than one day to calculate the threshold, my study area is a little bit big, and the data resolution is very fine. So is there any way to use more CPU cores during calculation? Because now the CPU has not been fully utilized, only 5% occupied. I also found from your manual that the default core is one core(shown in the following picture). Could you please kindly tell me what I need to do to improve the calculation efficiency?

Screen Shot 2022-01-12 at 11 47 45 PM
heroldn commented 2 years ago

In the climpact.ncdf.wrapper.r file that you execute you'll find a line specifying the number of cores (it should be set to FALSE by defaut, but you can change it to any number of cores that your CPU supports - so an 8 core CPU can handle specifying 8 cores).

cores=FALSE

JinZarrin commented 2 years ago

In the climpact.ncdf.wrapper.r file that you execute you'll find a line specifying the number of cores (it should be set to FALSE by defaut, but you can change it to any number of cores that your CPU supports - so an 8 core CPU can handle specifying 8 cores).

cores=FALSE

Thanks for your kind reply. Finally, I solved it, but the main problem is not how many cores we use, I adjust from one core to 16 cores, the did not change a lot. Finally, I found that the efficiency is mainly limited by the Disk read/write speed, I move my data to SSD, it improved about 20 times efficiency.