search

EvolvingLMMs-Lab / LongVA

Long Context Transfer from Language to Vision
Apache License 2.0
293 stars 16 forks source link

ValueError: Unknown model class ModelArguments(model_name_or_path='lmms-lab/LongVA-7B' #20

Open Wiselnn570 opened 3 weeks ago

Wiselnn570 commented 3 weeks ago

I got an error when running dpo.sh. I noticed that the script's entry point is llava/train/train_dpo.py. However, when I changed it to long/train/train_dpo.py, the bug occurred.

kcz358 commented 3 weeks ago

Hi, I have fixed this issue. Can you try again? We forgot to update the get_model with our current base_model name in the dpo training script

Wiselnn570 commented 1 week ago

Hi, I have fixed this issue. Can you try again? We forgot to update the get_model with our current base_model name in the dpo training script

Thank you, it’s working fine now. But if I need to add some video data during fine-tuning maybe to give the model new capabilities and train it together with the existing image data, how should I modify the fine-tuning script parameters? Does the current model support this type of training, or have you previously tried adding video data during fine-tuning? Thank you for your further explanation. I have already tried it, and I’m impressed by the model’s capabilities.

kcz358 commented 1 week ago

I think you can prepare the json file just like the image, just change the image to video and add video_path in the args