How do you perform model selection?

Sally-SH / VSP-LLM

Other

298 stars 25 forks source link

How do you perform model selection? #2

Closed xiabingquan closed 7 months ago

xiabingquan commented 7 months ago

Thanks for the great work.

After thoroughly readinging your codes, I'm wondering how you perform model selection. Did you perform model average on the several last checkpoints? Or did you evaluate each checkpoint on the validation set and them average the several best checkpoints?

I did not find any clue in the codes. Any reply will be appreciated, thanks!

Sally-SH commented 7 months ago

We selected the checkpoint that achieved the highest accuracy on the validation set. Thank you for your question.