Traceback (most recent call last):
File "./snippets/bug.py", line 18, in <module>
results = perplexity.compute(model_id='gpt2',
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/evaluate/module.py", line 462, in compute
output = self._compute(**inputs, **compute_kwargs)
File "/Users/pasquale/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--perplexity/8ab643ad86f568b7d1d5f7822373fa7401ff5ff0297ccf114b0ca6a33be96bc0/perplexity.py", line 179, in _compute
out_logits = model(encoded_batch, attention_mask=attn_mask).logits
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1074, in forward
transformer_outputs = self.transformer(
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 838, in forward
position_embeds = self.wpe(position_ids)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 162, in forward
return F.embedding(
File "/Users/pasquale/anaconda3/lib/python3.8/site-packages/torch/nn/functional.py", line 2233, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
Small snippet to reproduce the problem:
Here is the full trace: