is there plan to integrate ensemble step into the nni?

[joshua-xia]

What would you like to be added: I like to add the ensemble function into pipeline after hyper-opt

Why is this needed: the ensemble learning is very important to generate models

Without this feature, how does current nni work： current nni seems not support ensemble

Components that may involve changes: nni sdk need to provide ensemble learning algorithm library and config those algorithm into pipeline

Brief description of your proposal if any:

[chicm-ms]

Hi @joshua-xia , Thanks for binging model ensembling up. This can be added to our plan. It would be very helpful for us if you can describe your scenario and how would you like nni to help you with ensembling. Such as do you use decision tree or deep neural network ? what kind of ensemble methodologies (stacking, blending, etc) you are using? what kinds of features you want nni to have?

[joshua-xia]

I am using the stacking ensemble algorithm. for example the StackingClassifier in sklearn, and I think there are at least two kind of ways to integrate hyperopt and ensemble:

1. ensemble several models together and do hyperopt and ensemble the models in one pipeline at the same time, I can implements this case by using nested-search-space.
2. first to do hyperopt and then select the best models and then ensemble, this scenario need two phases of nni pipeline and integrate them together (pass the results from hyperopt phase to ensemble phase. nni need to support this kind of scenario.

[chicm-ms]

For the first way, if each first level model is tuned within one separate trial job, when doing the ensemble, how will you get other model's information in order to do ensemble in a trial job?
So are you planning to train multiple models + stacking model in a same trial job? each trial job run multiple first level model and stacking model (StackingClassifier )?

For the second way, we will do some investigation to see how nni can provide support in a general/elegant way for model ensembling.