search

haotian-liu / LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
https://llava.hliu.cc
Apache License 2.0
20.51k stars 2.27k forks source link

[Discussion] More data leading to lower indicators? #1364

Open uniquehou opened 8 months ago

uniquehou commented 8 months ago

Discussion

Hi, I tried LLaVA-1.5(13B), training an image classification task, with good results, but now I'm running into a problem. Before I used 250k data (finetune lora), the precision went up with more training data. Later, we added 200k data and the indicator dropped by 6-7% instead. We are now basically sure that the new data distribution is the same as before, so what else could be the reason for the decline?

awzhgw commented 8 months ago

same too. how to resolve it ?

awzhgw commented 8 months ago

@uniquehou how to resolve it ? i meet same problem

uniquehou commented 7 months ago

@uniquehou how to resolve it ? i meet same problem

Not yet. We're checking for overfitting.

AnnaGao0827 commented 1 month ago

Hi, i am dealing with finetuning overfit. I wonder if your 250k data follows the standard fine-tuning data format? And do you change your loss to classification loss,or keep the language modeling loss? Thanks a lot!