mlabonne llm-autoeval issues

mlabonne / llm-autoeval

Automatically evaluate your LLMs in Google Colab

MIT License

455 stars 77 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

token checks

#31 CrispStrobe opened 3 weeks ago
1
Fix broken links in README.md

#30 simekent closed 1 month ago
1
Novel errors when running notebook

#29 ann-brown opened 2 months ago
0
Add functionality for evaluating model safety/toxicity

#28 m-newhauser opened 2 months ago
1
how can I add a new benchmark? I'm trying to evaluate a text to sql model!

#27 tariksghiouri closed 2 months ago
1
Eqbench

#26 mlabonne closed 3 months ago
0
Legalbench

#25 Malikeh97 closed 3 months ago
0
Openllm fix

#24 mlabonne closed 3 months ago
0
Bug with lighteval on running from notebook

#23 ann-brown closed 3 months ago
1
Issues with BF16 models

#22 anakin87 closed 3 months ago
3
Ch dev

#21 chenhaodev opened 3 months ago
0
Proposal to Make LLM-AutoEval Repository Multi-Cloud Compatible

#20 adithya-s-k opened 3 months ago
1
Use lighteval for benchmarking

#19 burtenshaw closed 3 months ago
1
Use Colab notebook with private models

#18 burtenshaw closed 3 months ago
0
assert isinstance(pretrained, str)

#17 majacinka closed 3 months ago
2
How to run it locally?

#16 JamesKnight0001 opened 4 months ago
2
Only one GPU is used during the autoeval

#15 LMSPaul opened 5 months ago
0
Runpod.sh Refactor and new implementations.

#14 Steel-skull opened 5 months ago
4
Revert "Update runpod.sh"

#13 mlabonne closed 5 months ago
0
Revert "Update runpod.sh"

#12 mlabonne closed 5 months ago
0
Update runpod.sh

#11 CultriX-Github closed 5 months ago
0
Update runpod.sh

#10 CultriX-Github closed 5 months ago
0
Update runpod.sh

#9 CultriX-Github closed 5 months ago
0
Update runpod.sh

#8 Steel-skull closed 5 months ago
1
Feature request - Local GPUs

#7 maziyarpanahi opened 5 months ago
1
multiple GPUs works in colab

#6 tdolan21 opened 5 months ago
1
Docker setup for runpod

#5 johannhartmann closed 3 months ago
2
Why is `mmlu` benchmark commented out?

#4 sugatoray closed 3 months ago
3
Runpod says 'no longer any instance available'

#3 ncs-gobubble closed 5 months ago
4
Feature/time

#2 mlabonne closed 5 months ago
0
log running time?

#1 gblazex closed 3 months ago
3