aaronkl
commented
7 years ago Hi, here a brief intuition when to use which optimizer.
Fabolas is design for hyperparameter optimization of machine learning algorithm trained on large datasets. It obtains large speed ups compared to other methods such as SMAC, if single function evaluations are expensive (in the order of hours or days) and your datasets are large (usually more than 50000 data points). Since Fabolas uses a Gaussian process to model the objective function it only works in low dimensional (<= 10 dimensions) and continuous spaces.
SMAC uses a random forest instead of a Gaussian process and hence is able to model continuous and discrete input spaces. Compared to other Bayesian optimization frameworks it usually works better in high dimensional (discrete) configuration spaces where single function evaluations are relatively cheap (in order of minutes). SMAC can also run in parallel i.e. P-SMAC (see the SMAC docu for more infos) which can additionally speed up your optimization procedure if you have multiple CPU cores at your disposal.
cheers, Aaron
ytsaig
Thanks for publishing this package, I'm a fan of your group's work on AutoML, and found the corresponding paper (https://arxiv.org/abs/1605.07079) very interesting. I don't think this is the right forum for my question, but I'll ask it anyway. For someone looking to start with Bayesian hyperparameter tuning of large-scale ML models (where each training is on the order of 1 hour or more), and looking for a solid python framework to use, would you recommend RoBo/FABOLAS or SMAC (https://github.com/automl/SMAC3)? Obviously there are some modeling differences between the two approaches, but I found it hard to judge the maturity of each package and its applicability to general purpose hyperparameter optimization. Thanks!