search

artidoro / qlora

QLoRA: Efficient Finetuning of Quantized LLMs
https://arxiv.org/abs/2305.14314
MIT License
9.96k stars 820 forks source link

eval_loss or mmlu_loss #206

Open Duncan1115 opened 1 year ago

Duncan1115 commented 1 year ago

Hi, buddy！ Which loss is more important? eval_loss or mmlu_loss? In my opinion, eval_loss shows the accurary of the training dataset and mmlu_loss shows the accurary of the mmlu dataset. Am I right?

artidoro commented 1 year ago

That's correct! I usually look at eval loss and mmlu accuracy.