JohnSnowLabs / langtest

Deliver safe & effective language models
http://langtest.org/
Apache License 2.0
506 stars 40 forks source link

Fix Error in Accuracy Tests for NER Task #1115

Closed chakravarthik27 closed 2 months ago

chakravarthik27 commented 2 months ago

Always getting zero score due to y_true and y_pred are concatenate Example:

y_pred = pd.Series['O', 'ORG', 'O']
y_pred = pd.Series['lLOC', 'ORG', 'O']

Incorrect Extraction of classes:

classes = set(y_true + y_pred) 
classes 
['OLOC', 'ORGORG', 'OO'] 

Fixes in langtest/utils/util_metrics.py

if isinstance(y_true, list) and isinstance(y_pred, list):
    unique_labels = set(y_true + y_pred)
elif isinstance(y_true, pd.Series) and isinstance(y_pred, pd.Series):
    unique_labels = set(y_true.tolist() + y_pred.tolist())