Closed burtenshaw closed 3 months ago
This PR implements a benchmarking with lighteval within the Colab & Runpod configuration.
There is still some remaining work to unify the code base:
main.py
recommended_set.txt
@mlabonne It would be really cool to contribute this. Let me know what you think about integrating it with the existing benchmarks.
This PR implements a benchmarking with lighteval within the Colab & Runpod configuration.
There is still some remaining work to unify the code base:
main.py
The main features are tested:recommended_set.txt