search

evhub / bbopt

Black box hyperparameter optimization made easy.
Apache License 2.0
75 stars 8 forks source link

I wish i'd seen this... #12

Open microprediction opened 3 years ago

microprediction commented 3 years ago

I rather wish I'd noticed this before writing humpday.

evhub commented 3 years ago

@microprediction Haha—well, bbopt has a reasonably straightforward interface for adding additional backends, so you could add humpday as a backend if you wanted to use bbopt as your frontend. Additionally, all of bbopt is open-source and freely available, so you're also welcome to just steal things as long as you give credit.

evhub commented 3 years ago

@microprediction I just spent a bit of time playing around with what it might look like to add a humpday backend to bbopt, and it looks like the biggest roadblock is that it doesn't look like humpday's optimizers support the ability to pass in existing data (i.e. prior evaluations of the objective function). If that was fixed, I could probably integrate humpday into bbopt pretty easily, but without that feature, humpday doesn't really fit into bbopt's model—i.e. ability to stop and resume optimization from data on disk, ability to hot swap between different black box optimization backends, support for multiprocessing, etc.

microprediction commented 3 years ago

Quite true that's a limitation, and the intent of humpday is perhaps more limited than bbopt insofar as it is just supposed to provide an initial comparison. I'm not entirely sure that all the optimizers I call down to support state (e.g. dlib might not) but I'll take a look. It certainly would be useful to stop and restart.