search

8080labs / ppscore

Predictive Power Score (PPS) in Python
MIT License
1.12k stars 168 forks source link

Warning Message: F-score is ill-defined and being set to 0.0 in labels with no predicted samples. #35

Closed IliaIsaikin closed 4 years ago

IliaIsaikin commented 4 years ago

I am trying to calculate PPS for a classification task between two categorical columns and get this warning: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.

I included a screenshot of my code with some information on these two columns. Also, there are no missing values in the data.

Thank you for your help!

2020-08-17

FlorianWetschoreck commented 4 years ago

Hi Ilia,

the warning is nothing that you need to worry about. The result is still fine. At some point we will also try to deal with those warnings so that they are less confusing.

The warning tells you that the F1 is set to 0 in a subset where there are no predicted samples. Most likely, this happens for the class grade=G and loan_status=Charged Off because there are few data points within that subset. In addition, the cross-validation will split the dataset even further. In case that there will not be any row any more where loan_status=Charged Off when sklearn tries to calculate the F1 within the cross-validatio fold, sklearn will raise the warning.

I hope this helps?

IliaIsaikin commented 4 years ago

Hi Florian,

It sure helped! I was worried that something is wrong with my data, it’s good to know that the result is still good.

Thank you for creating this great library and sharing it with us. Looking forward to future updates and features!

FlorianWetschoreck commented 4 years ago

Great, happy to hear that!