Homogeneous ensembles

[ablaom]

Is it possible with this API to build a homogeneous ensemble of learners where each learner shares the same hyper parameters as all the others. So, for example, if I tune this composite model, I tune the hyperparameters in tandem, rather than independently?

[alegonz]

@ablaom Please let me first check if I understood correctly the problem setting. Given two learners of the same kind, each with a parameter alpha:

• learner 1 with parameter alpha = 1, 2, 3...
• learner 2 with parameter alpha = 1, 2, 3...

you want to tune the combinations "along the diagonal" (so to speak) of the search space:

• learner1_alpha = 1, learner2_alpha = 1
• learner1_alpha = 2, learner2_alpha = 2
• etc

and not combinations like

• learner1_alpha = 1, learner2_alpha = 2
• learner1_alpha = 2, learner2_alpha = 1
• etc

right?

If the above is correct, yes, you can tune homogeneous ensembles with baikal. However, that's actually does not have much to do with baikal, it is rather about the API of sklearn's GridSearchCV. (baikal models are at present meant to be tuned with GridSearchCV and the SKLearnWrapper). Fitting a homogeneous ensemble of learners is a matter of given the appropriate param_grid to GridSearchCV. For the example above it would be something like:

# note the list of dicts
homogeneous_param_grid = [
    {"learner1__alpha": [1], "learner2__alpha": [1]},
    {"learner1__alpha": [2], "learner2__alpha": [2]}
]

# this will also compute crossed combinations
full_param_grid = {
    "learner1__alpha": [1, 2],
    "learner2__alpha": [1, 2]
}

[ablaom]

Thanks for your detailed response. This does answer my question!