TrAdaBoost can predict_proba or not

[jnuvenus]

Can Tradaboost algorithm use the predict_proba() method to output the probability values of each sample belonging to the positive and negative classes in binary classification problems?

[antoinedemathelin]

Hi @jnuvenus, Thank you for your interest in the Adapt library! There is no predict_proba method implemented on the Tradaboost object but you can access the predict_proba method of each estimator in model.estimators_. Here is a little example to compute the weighted average probabilities among Tradaboost estimators (Note that the authors of Tradaboost suggests to use only the last half estimators for prediction):

import numpy as np
from sklearn.linear_model import LogisticRegression
from adapt.utils import make_classification_da
from adapt.instance_based import TrAdaBoost

Xs, ys, Xt, yt = make_classification_da()
model = TrAdaBoost(LogisticRegression(), n_estimators=10, Xt=Xt[:10], yt=yt[:10],
                   verbose=0, random_state=0)
model.fit(Xs, ys);

N = len(model.estimators_)
probas = [m.predict_proba(Xt) for m in model.estimators_[int(N/2):]]
probas = np.stack(probas, axis=-1)

weights = np.array(model.estimator_weights_)
weights = weights[int(N/2):]
weights /= weights.sum()

weighted_avg_probas = probas.dot(weights)
weighted_avg_probas

Thanks to your question, we will add the predict_proba feature for Tradaboost in the next release of Adapt