I am trying to calculating surprisal value by feeding in a txt file with about 5000 sentences. But there is a error message I encounter: IndexError: index out of range in self Can anyone help with this issue?
Thank you!
Who can help?
No response
Information
[ ] The official example scripts
[ ] My own modified scripts
Tasks
[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)
Reproduction
Here is the code:
Here is the error message:
Expected behavior
I would like to have the surprisal value for each word for the whole text file.
System Info
I am trying to calculating surprisal value by feeding in a txt file with about 5000 sentences. But there is a error message I encounter: IndexError: index out of range in self Can anyone help with this issue?
Thank you!
Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Here is the code:
Here is the error message:
Expected behavior
I would like to have the surprisal value for each word for the whole text file.