search

scikit-learn-contrib / DESlib

A Python library for dynamic classifier and ensemble selection
BSD 3-Clause "New" or "Revised" License
479 stars 106 forks source link

predict_proba from different methods are the same #184

Closed jayahm closed 4 years ago

jayahm commented 4 years ago

Hi.

For each DCS method in the package, I used .predict_proba to produce the probability scores.

For example,

ola.predict_proba(x_test)
lca.predict_proba(x_test)

and so on.

I wonder why the scores produced are the same for all methods?

each of them suppose to chose the score produced by the selected classifier, right?

Note: Accuracy are not same.

Menelau commented 4 years ago

Hello @jayahm

DCS methods do use the probability estimates of the selected base classifier when calling the predict_proba. And I can guarantee you that the predict_proba of different DCS models gives different results (as they may differ in the selected classifier for each example).

I created one example that shows the predict_proba of different DCS methods gives different outputs. You can check the example in the following gist: https://gist.github.com/Menelau/6a5b1fb82cb3ef5618c358a8a2fbde99

In your case, it could be a scenario where all DCS methods agree that the same classifier is the competent one. So that the probabilities estimates are the same.

jayahm commented 4 years ago

Maybe I misunderstood something. Thank you for the clarification.