However, it looks like it is not included as part of your code and instruction to use it are missing.
In the latest branch there are several changes in the API of lm-eval-harness.
Can you include it as part of your code or point to the specific commit we should use your code
Any more instructions will be really helpful.
Hello Authors,
For the downstream evaluation we understand [lm-eval-harness]https://github.com/EleutherAI/lm-evaluation-harness/tree/master/lm_eval) is being used.
However, it looks like it is not included as part of your code and instruction to use it are missing.
In the latest branch there are several changes in the API of lm-eval-harness.
Can you include it as part of your code or point to the specific commit we should use your code Any more instructions will be really helpful.