BERT/T5 dataset handles already corrupted indices incorrectly.
GPT tokenizer vocab size does not include special tokens.
In Eval Harness evaluation, the last token of the target is unconditionally truncated. This results in wrong evaluation metrics and is actually not such a small problem.
This PR fixes these issues. Since the changes are so small, I didn't bother creating 3 PRs.
Problems:
This PR fixes these issues. Since the changes are so small, I didn't bother creating 3 PRs.