search

emacs-ess / ESS

Emacs Speaks Statistics: ESS
https://ess.r-project.org/
GNU General Public License v3.0
620 stars 162 forks source link

ESS: Memory allocation warning before running distance matrix #591

Closed tmbsundar closed 6 years ago

tmbsundar commented 6 years ago

I am running ESS from WSL in Windows 10. (no direct linux because it's a workplace image.) We have R Studio as well. I prefer to use Emacs/ ESS.

The issue I faced is this: While running hcluster and pamk clustering methods, we need to calculate the distance matrix. I used daisy from cluster package and pamk from the fpc package. In both the cases, the R inferior process ran for some good amount of time (say 20-30 minutes or so.) and failed with a Segmentation fault error.

There are around 45000 rows and 60 variables. The RAM of the machine is 8 GB.

I ran them in RStudio. RStudio immediately gave me a warning that "cannot allocate vector of size 7.7 Gb" and aborted the invocation.

The question here is this: Is there a way to improve ESS behavior like RStudio. Am I missing any packages/ settings or plug ins?

Time is not the only problem. After such a crash, the entire R session loses all variables etc., and the session becomes unresponsive and only option is to kill all, exit and then restart Emacs and ESS. So, it will be nice to have a forewarning from ESS and an option to abort and return to the base process.

I googled a lot and did not find any solution. Hence, raising the issue here. Thanks.

jabranham commented 6 years ago

Are you running Rstudio from WSL in Windows 10 as well? That error message is generated by R, not Rstudio. ESS and RStudio should get the same information from R.

vspinu commented 6 years ago

Indeed. It's probably just accidental that crash hapend with ESS. Probably at the moment you run the command with ESS there was memory and computation started, but then a segmentation fault was triggered (which, btw, might not even be memory related).

tmbsundar commented 6 years ago

Thanks for the replies.

@jabranham I tested RStudio not from WSL, but from the Anaconda RStudio installation directly from Windows 10.

@vspinu Also, I have encountered this on a few occasions. Hence, this seems like it is not a accidental happening. One explanation could be that RStudio runs from Win 10 directly vs ESS from WSL.

But, if it's the direct message from R in both the instances, the behavior should be similar. I will try to reproduce with screenshots and post if that helps.

tmbsundar commented 6 years ago

@jabranham You are right. This seems to be an issue with WSL and not from ESS.

I installed RStudio in WSL and ran the same code. The behavior was same as that of ESS. That is, RStudio ran for some time and gave segfault error with core dumped.

I reran the experiment, with installing Emacs and ESS in direct Windows 10 and ran the same code from within ESS. This time the early memory allocation warning message was thrown from ESS/R.

There seems to be some issue with WSL picking up the warning from Rkernel in both ESS and RStudio cases.

Apologize for raising this issue without testing both on the same platform. But, updating here so that, this is taken to a logical closure and useful for someone who faces the issue in the future.

Thanks for your support and the great ESS product. Thanks.

P.S: Changed the title of the issue to reflect this appropriately.