Closed pgoodling-usgs closed 2 years ago
752/420 = 1.8
is close to a 2-fold speedup. 2-fold is the theoretical optimum speedup and is unlikely to happen.
On your machine, what are the results of parallel::detectCores()
?
The issue with the 8 core run is that using n_fold_hyperparameter <- 3
limits the number of parallel works to be 3 (see next paragraph tho). Using 8 should not really give you >3-fold speedup and, if this is on windows, there might be some jumping around of processes across cores that might make is slower.
If you want to use more cores, there is an option in the control function to parallel_over = "everything"
. The blog post that you reference describes the different in the options and gives some intuition to which option is best (spoiler: it depends on what you are doing).
Aha- I didn't realize that the # of cores would be limited to n_fold_hypeparameter
, I thought it would be the overall folds in cur_cv_folds
.
With 2 cores and parallel = "everything"
the runtime is 559 seconds, which is slower than the 420 seconds for parallel="resamples"
. With 8 cores parallel = "everything"
the runtime is 801 seconds, which is slower than the 558 seconds for parallel="resamples"
(and even slower than the unparallelized ~750 seconds).
When I run parallel::detectCores()
, I get 8. So I probably shouldn't expect 8 cores to fully be engaged, right?
I guess overall the speedups are simply difficult to predict and are more related to underlying R / windows features. You've documented this in various blog posts so it might not really count as an issue with tune
.
This issue has been automatically locked. If you believe you have found a related problem, please file a new issue (with a reprex: https://reprex.tidyverse.org) and link to this issue.
Hello,
I'm working on parallelizing across many resamples with a large dataset. I ran into some of the issues described in #376 and #384 , and was glad to see that in Tune 0.2.0 there was a fix (described in #397). I reproduced the example in #397 below, though switched it to a ranger random forest model.
With 5,000 rows and 10,000 columns also see that with a single non-parallel worker, the usage peaks at ~3.2-3.5 GB and that with a 8 parallel workers each peaks at ~1.2GB-1.5GB (for a total memory usage between 9GB and 10GB).
However, the decrease in execution time isn't quite as advantageous as I would expect. With 24 resamples and 8 cores, each core should work through 3 iterations of the model fitting. Based on this post it seems difficult to estimate the time savings a priori, but I still expected a greater efficiency than going from 752 seconds elapsed to 558 seconds elapsed.
In Chapter 10 of the Tidymodeling book it says that
So I expect at least a 2-fold increase in computation speed, right? Is there something about the 0.2.0 upgrade that would have affected this? Is this typical behavior or am I missing something in my configuration or my understanding?
Created on 2022-04-14 by the reprex package (v2.0.1)
UPDATE: I ran this code using just 2 cores and wound up with a completion time of 420 seconds and in each core the memory peaked around 2.5 GB. It running faster on 2 cores than on 8 cores is certainly unexpected and goes against the statement that "Using more cores will still reduce the time it takes to complete the task; there are just diminishing returns for the additional cores."
Here is my SessionInfo():