How does fitbert handle answers that have multiple tokens?

writer / fitbert

Use BERT to Fill in the Blanks

https://pypi.org/project/fitbert/

Apache License 2.0

82 stars 14 forks source link

How does fitbert handle answers that have multiple tokens? #22

Closed mannt-jp closed 2 years ago

mannt-jp commented 3 years ago

If the answer contains multiple tokens (for example containing multiple words, or even a whole sentence), how does fitbert compute the cummulative probability of this kind of answer. Because MLM outputs the probabilities for every single tokens, not for groups of tokens.

sam-writer commented 2 years ago

I would use BART for this, see https://github.com/writerai/fitbert/issues/19#issuecomment-913618892