search

YiyangZhou / POVID

[Arxiv] Aligning Modalities in Vision Large Language Models via Preference Fine-tuning
Apache License 2.0
74 stars 3 forks source link

About CHAIR evaluation #5

Open Kizna1ver opened 7 months ago

Kizna1ver commented 7 months ago

Hi, thank you for your great work. Can you provide more details about the CHAIR evaluation? Are all COCO val2014 images used for evaluation? And what prompt is used for inference? It would be appreciate if you can provide the image_ids used for evaluation.

YiyangZhou commented 7 months ago

Hi, thank you for your great work. Can you provide more details about the CHAIR evaluation? Are all COCO val2014 images used for evaluation? And what prompt is used for inference? It would be appreciate if you can provide the image_ids used for evaluation.

Hi, thank you very much for your interest in our work. Our test set consists of 500 randomly sampled images from val2014. The specific prompt used is: "Describe this image." We didn't save the specific IDs at that time, but we plan to update with a full parameter version in the future. We will retest CHAIR and provide the test images then.

CrossLee1 commented 4 months ago

@YiyangZhou Hi, have you released the specific IDs and the new CHAIR performance. Thanks ~