search

jbrea / BayesianOptimization.jl

Bayesian optimization for Julia
Other
91 stars 17 forks source link

Hybrid continuou and discrete BO #26

Open mohamed82008 opened 3 years ago

mohamed82008 commented 3 years ago

Hi, thanks for this great package. Do you have hybrid BO with discrete and continuous variables planned? I recently came across https://arxiv.org/pdf/2106.04682.pdf and it discusses an interesting approach so I thought I would ask here. Thanks.

jbrea commented 3 years ago

That looks like a cool paper, thanks! I think this is one direction to go with this package. Other directions include 1) make it work better in higher dimensions (there is quite some recent literature on scaling BO) 2) have better default options (e.g. using some of the heuristics used by the python BO package dragonfly).

Unfortunately I don't have much time in the next few months to add substantial features to this package. But PRs are highly appreciated :)

mohamed82008 commented 3 years ago

Great directions! I think these would make great GSoC projects. Also, I see that you depend on NLopt. Is it possible to use something like Nonconvex.jl for the optimization?

jbrea commented 3 years ago

Yes. Replacing the optimiser should be a simple thing. At the time I wrote the optimisation code I only considered Optim and NLopt and found the latter preferable. Are there compelling reasons to switch to Nonconvex.jl?

mohamed82008 commented 3 years ago

Nonconvex makes it easy to switch between many optimization packages. Also it will make it easier to incorporate inequality and equality constraints into the optimization. Nonconvex also uses Zygote by default so that might be more efficient in some cases. If you are ok with the direction, I will be happy to open a PR.

jbrea commented 3 years ago

Yes, that sounds great. Thanks!

LeviManring commented 2 years ago

Any update on this aspect? https://arxiv.org/abs/1805.03463 also has some interesting ideas on this front

jbrea commented 2 years ago

I don't have currently the time to work on this but I wrote a GSoC proposal.

@LeviManring I think the method in the paper you mention performs typically worse than HyBO (the paper mentioned by @mohamed82008; see its supplementary section).