search

automl / SMAC3

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
https://automl.github.io/SMAC3/v2.1.0/
Other
1.04k stars 220 forks source link

SMAC in parallel mode and impact of #cores on performance #1082

Closed ndangtt closed 7 months ago

ndangtt commented 8 months ago

Hello! When I run SMAC in parallel mode, assuming that I have a fixed budget of 100 trials and that each configuration is fairly expensive to evaluate, would SMAC always utilise all the given #cores, and would there be difference in term of performance (of the best configuration found) if I run the tuning on 36 cores vs 10 cores vs 1 core?

benjamc commented 7 months ago

Hi, SMAC should utilise all given workers. Probably, there will be a difference in performance because of the order the surrogate model is fitted. Fitting the model in a different order also results in different proposals for configurations. However, we did not benchmark how much the performance would change depending on the number of workers. You can specify after how many trials the surrogate model is retrained in the config selector.

ndangtt commented 7 months ago

I see, thank you for your reply!