The annotator can't handle sentences longer than 512 tokens. - Githubissues

spraakbanken / sparv-sbx-word-prediction

Plugin for applying bert masking

MIT License

0 stars 0 forks source link

The annotator can't handle sentences longer than 512 tokens. #6

Closed kod-kristoff closed 8 months ago

kod-kristoff commented 9 months ago

split sentence

kod-kristoff commented 9 months ago

https://stackoverflow.com/questions/70520725/runtimeerror-the-expanded-size-of-the-tensor-585-must-match-the-existing-size/70626497#70626497