Open nikosavola opened 1 year ago
Seems cool! Would you be interested in making an initial PR? We'd be happy to shepherd it after :)
general black-box optimisation.
This would probably have to be the global and local derivative-free algorithms. The ones that require gradients probably won't work without them, and it's hard to supply those gradients to Tune (you'd have to have a setup similar to meta learning to collect e.g. d loss / d hyperarameter
).
It's also unclear to what degree algorithms like NLOpt would be for Tune. Algorithms like BayesOpt are designed to be sample efficient, by contrast, I am a little afraid that NLOpt algorithms may assume evaluation of the objective function f is cheap.
It is also a little unclear to me to what degree the constraints will be useful, especially non-linear constraints. Perhaps I am intending to set a non-linear bound on the complexity of my models?
e.g. channels^3 + layers^2 < 500
Anw, I think I can give this a shot.
Description
NLopt support could be added to the search algorithms to increase the amount of available optimisation algorithms and suitability for general black-bok optimisation.
There exists a Python API with a common interface through
nl.opt
to which support could be added.Use case
Enabling using Ray Tune as a manager for black-box optimisation with NLopt.