automl / RoBO

RoBO: a Robust Bayesian Optimization framework
BSD 3-Clause "New" or "Revised" License
482 stars 133 forks source link

RoBo in discrete spaces #110

Closed MalteEbner closed 5 years ago

MalteEbner commented 5 years ago

Is it possible to use RoBo with discrete input spaces? An example would be choosing different gradient descent strategies for a neural network. It is sufficient if only an index e.g. element {0, 1, 2, 3} is chosen, while simultaneously optimising real input parameters

aaronkl commented 5 years ago

Unfortunately, RoBO only supports continuous input spaces. A simple and dirty fix would be to use a one-hot encoding for categorical inputs and simply round integers inside the objective function. Obviously, this will deceive the model and it might need a few more function evaluations to capture this. Alternatively, you could checkout emuki which is able to handle discrete parameters. For example see https://github.com/amzn/emukit/blob/master/notebooks/Emukit-Categorical-with-Tensorflow.ipynb