search

Liuziyu77 / MIA-DPO

Official implement of MIA-DPO
Apache License 2.0
36 stars 1 forks source link

dpo data generation #1

Closed lyklly closed 6 days ago

lyklly commented 6 days ago

i notice '/mnt/petrelfs/baitianyi/mmdpo/datasets/mia/llava187k.json' in mix_data.ipynb, however i can't find how to get this file.in the paper, it says that you random sample in 665k data. but when i do that, the data structure is very different from the data structure you used. can you tell me what to do?

what's more, notice "export model_name_or_path=/mnt/hwfile/mllm/liuziyu/finetune_LLaVa/merge_lora_RLHF_llava_mix_textvqa_20k_coco_25k" in train_dpo_multi.sh, does this mean ShareGPTVideo/LLaVA-Hound-SFT or you finetune the model in certain dataset?

Liuziyu77 commented 6 days ago

llava187k.json contains 187k data sampled from LLaVa665k，you can sample the same or a different amount of data yourself.

When generating data and using different DPO models, we may encounter different data structures, so feel free to adjust the data structure as needed. However，when using our code to perform DPO on the model, your DPO data needs to align with our DPO data format, which is different from the llava665k data format

As for model_name_or_path, you can download liuhaotian/llava-v1.5-7b locally and use the path to this model.

lyklly commented 6 days ago

thanks a lot, clear answer!