Fixes the incorrect token prediction distribution from _all_scores_for_token() in sequence_tagger_model.py

flairNLP / flair

A very simple framework for state-of-the-art Natural Language Processing (NLP)

https://flairnlp.github.io/flair/

Other

13.85k stars 2.09k forks source link

Fixes the incorrect token prediction distribution from _all_scores_for_token() in sequence_tagger_model.py #3449

Open mdmotaharmahtab opened 5 months ago

mdmotaharmahtab commented 5 months ago

This PR fixes the issue #3448 . Previously, due to incorrect length calculation for each sentence in the batch, the returned tag probability distribution for each token was incorrect (from the _all_scores_for_token() function in sequence_tagger_model.py). This PR makes a small change in _all_scores_for_token() function to correctly compute the length.

alanakbik commented 5 months ago

@mdmotaharmahtab thanks a lot for this PR!

@whoisjones can you review?

mdmotaharmahtab commented 5 months ago

I have examined the failing unit tests and found that they also failed on the master branch (so rebasing on the master did not help). They may not be related to this PR. Do I need to work on these tests for this PR? @whoisjones @alanakbik

MdMotahar commented 1 month ago

Hello. Want to know if this PR will be reviewed any time soon? @whoisjones @alanakbik