question on "Revisit Multi-Choice Question Benchmarks"

deepseek-ai / DeepSeek-LLM

DeepSeek LLM: Let there be answers

https://chat.deepseek.com/

MIT License

1.33k stars 87 forks source link

question on "Revisit Multi-Choice Question Benchmarks" #18

Closed imhmhm closed 7 months ago

imhmhm commented 7 months ago

Thanks so much for sharing the findings and insights about "Multi-Choice Question Benchmarks", I have a quick question about the 20 million Chinese MC data leading to overfiting without generalizing to other tasks, are the data composed of questions with pure options OR with sort of explanations in the answers?

Thank you again for your great work!

DeepSeekPH commented 7 months ago

Pure options