Expose method/class to interact with the metalearner. A use case: get initial configuration only

[gomerudo]

The problem

There is a use case where users would like to explore the output from the meta learning suggestions for a given dataset. This is, given a dataset I would like to obtain a configuration (suggestion) from the metalearner without running the optimization procedure.

The current implementation

I noticed that in the autosklearn/smbo.py file the method get_metalearning_suggestions() is somehow in charge of this operation however it comes hard to figure out the way to call it due to the complexity of the code (the method is only called in the methdo run_smbo() in smbo.py and then the latter method is called in the _fit() method of automl.py where too many steps co-exist).

The requested solution

This ticket then requests a method (possibly a different class) to obtain a configuration based on meta learning without executing the SMAC algorithm to tune the hyper-parameters (i.e., add an extra layer of abstraction where the meta learner can be differentiated from the optimizer).

Temporal workaround (check feasibility first)

In the meantime, I think (intuitively) that it is possible to add to the documentation (and to this ticket) one snippet to achieve this task with the current methods available in the library (an expert on the code may figure it out much faster that inexperienced auto-sklearn users).

Let us know if this sounds like a feasible improvement!

[mfeurer]

Thanks for bringing this up. This would indeed be a great addition, but I'm afraid we won't get to implement that soon by ourselves. Also, there's no way to achieve what you want with the current code base. A brief workaround would be to replace self.runhistory_, self.trajectory_ = _proc_smac.run_smbo() but a call to get_metalearning_suggestions() and see if that works.

In the long run we would appreciate if you would help us out by adding this feature to Auto-sklearn, and we'd be happy to help you restructuring the code to achieve this.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs for the next 7 days. Thank you for your contributions.