microsoft / LLaVA-Med

Large Language-and-Vision Assistant for Biomedicine, built towards multimodal GPT-4 level capabilities.
Other
1.48k stars 181 forks source link

Json labels for the three VQA datsets #24

Open tmp12316 opened 10 months ago

tmp12316 commented 10 months ago

Thank you so much for your great work! Are there some official scripts to process VQA labels into conversation format for the 3 VQA datasets? Or could you please provide the pre-processed json files of them? Thanks.

LiangXin1001 commented 8 months ago

Thank you so much for your great work! I also encountered this issue. Although I can download all the images, I still don't know how to generate the contents for the JSON files, such as test.json and train.json. Moreover, in the files from Hugging Face you provided, I found that the questions are all of a type that only requires a 'yes' as an answer. Can such questions be included as content in the JSON files? image

LiangXin1001 commented 8 months ago

And also, when I input the model_vqa_med.py code to GPT-4, it suggested that the input JSON file should look like this: { "id": "unique_question_identifier", "conversations": [ { "from": "user", "value": "text question" }, { "from": "gpt", "value": "expected answer" } ], "image": "optional_image_path.jpg" } Is this correct?

Eldo-rado commented 5 months ago

Thank you so much for your great work! I also encountered this issue. Although I can download all the images, I still don't know how to generate the contents for the JSON files, such as test.json and train.json. Moreover, in the files from Hugging Face you provided, I found that the questions are all of a type that only requires a 'yes' as an answer. Can such questions be included as content in the JSON files? image

Hi, I meet the same issue, have you solved it? 😊

wangpengyu-debug commented 3 weeks ago

is there any differences when processing the data to { "id": "unique_question_identifier", "conversations": [ { "from": "user", "value": "text question" }, { "from": "gpt", "value": "expected answer" } ], "image": "optional_image_path.jpg" } this format?