[feature] editor.suggest returns likelihood for each suggestions

marcotcr / checklist

Beyond Accuracy: Behavioral Testing of NLP models with CheckList

MIT License

2.01k stars 204 forks source link

[feature] editor.suggest returns likelihood for each suggestions #73

Closed jlema closed 3 years ago

jlema commented 3 years ago

Currently editor.suggest returns words or tuples ordered by likelihood. It would be great to be able to return the actual likelihood to compare likelihoods between suggestions. This could be done by passing the meta=True argument

Sample usage:

sug = editor.suggest('this is a tiny {mask} dog', meta=True)
print(sug.data[0], sug.meta[0])
print(sug.data[0], sug.meta[0])
'cute', 0.98
'yappy', 0.86

marcotcr commented 3 years ago

Implemented in 46b7263, but with parameter return_score and different return format (returns tuple if True)