Implemented eval scripts for VQA on the Clevr and Qlevr datasets, and tested to be working with model-pathagi-collective/mistral-7b-oh-siglip-so400m-finetune-lora and model-baseteknium/OpenHermes-2.5-Mistral-7B
Note that I assume the question file to be in the same format as found in the downloaded datasets.
Question file questions_test.json, each question references an image with an id that corresponds to images in 2d_scene/test/full_images/2d_full_test_<image-id>.png or 3d_scene/test/images/3d_test_<image-id>.png.
Implemented eval scripts for VQA on the Clevr and Qlevr datasets, and tested to be working with model-path
agi-collective/mistral-7b-oh-siglip-so400m-finetune-lora
and model-baseteknium/OpenHermes-2.5-Mistral-7B
Note that I assume the question file to be in the same format as found in the downloaded datasets.
Clevr
CLEVR_v1.0/questions/CLEVR_test_questions.json
, each question references an image found inCLEVR_v1.0/images/test
.Qlevr
questions_test.json
, each question references an image with an id that corresponds to images in2d_scene/test/full_images/2d_full_test_<image-id>.png
or3d_scene/test/images/3d_test_<image-id>.png
.