IndexError: list index out of range While generating paraphrase

[tuhinjubcse]

Traceback (most recent call last):
File "generate_paraphrases.py", line 478, in cli_main() File "generate_paraphrases.py", line 474, in cli_main main(args) File "generate_paraphrases.py", line 374, in main hypos = task.inference_step(generator, models, sample, prefix_tokens) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/fairseq/tasks/fairseq_task.py", line 265, in inference_step return generator.generate(models, sample, prefix_tokens=prefix_tokens) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(args, kwargs) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/fairseq/sequence_generator.py", line 113, in generate return self._generate(model, sample, kwargs) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(args, kwargs) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/fairseq/sequence_generator.py", line 152, in _generate encoder_outs = model.forward_encoder(encoder_input) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 26, in decorate_context return func(*args, kwargs) File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/fairseq/sequence_generator.py", line 542, in forward_encoder return [model.encoder(encoder_input) for model in self.models] File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/fairseq/sequence_generator.py", line 542, in return [model.encoder(*encoder_input) for model in self.models] File "/home/tuhin.chakr/yes/envs/prismenv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, kwargs) File "generate_paraphrases.py", line 159, in forward penalties = make_word_penalties(line=line, vocab=self.vocab_set, mapx=self.mapx) File "generate_paraphrases.py", line 75, in make_word_penalties next_word = uline + line2[ii+prefix_len][0] # just penalize starting a word, not continuing it IndexError: list index out of range

Here is my test.src file https://drive.google.com/file/d/1qi3xCRmxDE3wqS9jFAzQxLyHSJP30cnO/view?usp=sharing Here is my test.tgt file https://drive.google.com/file/d/1g7wykUQfsGeismMPbAxPUhZr7acXs5Ba/view?usp=sharing

[thompsonb]

Your input is almost 3M lines - please figure out what sentence is causing the error if you want me to try to debug it.