chigkim / Ollama-MMLU-Pro

Apache License 2.0
60 stars 13 forks source link

Correct Random Guesses, division by zero error #5

Closed korziner closed 2 hours ago

korziner commented 3 hours ago
Finished testing computer science in 13 hours 32 minutes 39 seconds. Total, 294/410, 71.71% Random Guess Attempts, 0/410, 0.00% Correct Random Guesses, division by zero error Adjusted Score Without Random Guesses, 294/410, 71.71% Finished the benchmark in 13 hours 32 minutes 45 seconds. Total, 294/410, 71.71% Token Usage: Prompt tokens: min 1448, average 1591, max 2043, total 547442, tk/s 11.23 Completion tokens: min 61, average 242, max 2048, total 83171, tk/s 1.71 Markdown Table: overall computer science
71.71 71.71

Report saved to: eval_results/qwen2-5-32b-instruct-q4_K_M/report.txt

chigkim commented 2 hours ago

That's because there was no random guess, and you can't divide by 0. You can just ignore it.