automl / neps

Neural Pipeline Search (NePS): Helps deep learning experts find the best neural pipeline.
https://automl.github.io/neps/
Apache License 2.0
61 stars 13 forks source link

[Optimizers] Programmatic access to Optimizer's capabilities/requirements #119

Open eddiebergman opened 4 months ago

eddiebergman commented 4 months ago

Right now, there's no way to programmatically know that an optimizer supports multi-fidelity, multi-objective, cost-awareness or requires priors to be set.

This makes parametrized testing quite difficult as a result. What would be ideal is that from SearchMapping, we can traverse through it and get the capabilities of the optimizer, preferabbly from cls.Supports.fidelity, cls.Supports.multi_objective, cls.Supports.priors`, etc...

This would also be useful for CLI and documentation purposes, as right now this would all require manual specification and given then umber of optimizers, this would lead to a lot of inconsistencies building up over time if we were to do it manually.

One partial issue is that we use partial in the SearchMapping, meaning that we would have to go partial._inner_function.Support.blah (or whatever its called). Not a huge issue unless the partial actually changes the capability of the optimizer.