Open sepiatone opened 3 months ago
There has been a refactor of lm-eval-harness a few months ago. The starter colab notebook for lm-eval-harness, linked to from this section has to be slightly updated to reflect this.
Instead of
!python lm-evaluation-harness/main.py \ --model gpt2 \ --tasks truthfulqa_gen \ --device 0
we have
!lm-eval --model hf --model_args pretrained=EleutherAI/Pythia-70m --tasks truthfulqa_gen --device 0
thanks, I'd be glad if you submitted a PR!
will do that!
There has been a refactor of lm-eval-harness a few months ago. The starter colab notebook for lm-eval-harness, linked to from this section has to be slightly updated to reflect this.
Instead of
we have