Azure / doAzureParallel

A R package that allows users to submit parallel workloads in Azure
MIT License
107 stars 50 forks source link

If I use chunkSize=1, is there a new R session per iteration of the foreach loop? #347

Closed slavarazbash closed 5 years ago

slavarazbash commented 5 years ago

@MicahMcKittrick-MSFT re-posting here as requested on MSDN.

I intend to use Azure Batch to run R in parallel with the do AzureParallel package. I intend leave the default chunkSize = 1.

Does it start a new R session for each iteration of the foreach() loop?

I do want it to start a new R session for each iteration of the foreach() loop. If the answer to the above question is "no", how can I make it restart the R session for each iteration?

Here is the link to the documentation of the chunkSize option, with example code.

https://github.com/Azure/doAzureParallel/blob/master/docs/80-performance-tuning.md#using-the-chunksize-option

I would like to start a new R session because it seems to be the best way to clear out R's memory.

mimckitt commented 5 years ago

@slavarazbash I will let @brnleehng continue the conversation here or on MSDN

slavarazbash commented 5 years ago

Answered by Brian Hoang on MSDN. Yes, it does.