Manually set initial values to Bayesian Optimization?

[jussihi]

Is there a way to manually give at least some of the initial values for the BO?

Currently I'm using struct init_randomsampling in my Params struct, but as its name implies, the samples are randomly drawn.

Let's say I have made a good guess for the solution of my problem and I would like to include this guess to my init samples. How could I do this?

[costashatz]

You just need to create a new initialization structure. You can take the RandomSampling (see here) as an example and create your new one. Then make sure to pass the new initialization scheme to the BOptimizer: you can do this by bayes_opt::BOptimizer<..., init_fun<MyNewInit>,...> when creating your BOptimizer object.

[jussihi]

Thank you @costashatz . I was able to do it with your instructions. Small note, the right name for the initialization function in BOptimizer's constructor is initfun, not init_fun.

Since LIMBO is licensed under CECILL-C which is somewhat similar to LGPL and obligates me to publish any changes/improvements made to the library itself, do you want me to publish the changes I've made to achieve my initial requirement (setting manual initial values / good guess to BOptimizer)? If so, how would you prefer me to publish them?

[nickanthony-dgl]

@jussihi . I am also interested in being able to specify initial values for the optimization. Would you be willing to post your implementation here?

[jussihi]

@jussihi . I am also interested in being able to specify initial values for the optimization. Would you be willing to post your implementation here?

I chose to use different optimization library, so I don't have the implementation with me anymore, but it was something like this: In random_sampling.hpp, I added a method set_init_sample into Params::init_randomsampling. For this to work, the init_randomsampling should be changed from BO_PARAM to DYN_PARAM IIRC. Then I created a new copy of struct RandomSampling, named it RandomSamplingWithInitSample and used the init_randomsampling's initial values set from set_init_sample before the for loop.

In code, this could then be used as follows:

Params::init_randomsampling::set_init_sample(initial_guess);

limbo::bayes_opt::BOptimizer<Params, limbo::initfun<limbo::init::RandomSamplingWithInitSample<Params>>> boptimizer;
boptimizer.optimize(Eval());

I'm sorry, I don't have the exact implementation anymore, this is just how I remember implementing it. Hope this helps!

[jussihi]

I will also close this issue, since I got my question answered long ago. Too bad I was not able to contribute since I never received any instructions for code review, etc.

[costashatz]

Too bad I was not able to contribute since I never received any instructions for code review, etc.

Hello @jussihi. Sorry for the very late reply. You are welcome to contribute any code you wish.

Feel free to make a PR and we will review it.