Do you mind providing the exact command or script used to produce the evaluation results?

[young-geng]

Please follow the serving and evaluation documentation of EasyLM to run the evaluation we did. A much simpler way is to use lm-eval-harness. Just remember to turn off the fast tokenizer as transformers fast tokenizer has a bug,