search

Q-Future / Q-Instruct

②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.
https://q-future.github.io/Q-Instruct/
Other
205 stars 9 forks source link

About IQA evaluation code #4

Closed Qiulin-W closed 12 months ago

Qiulin-W commented 12 months ago

Hi,

Great work! I'm not clear about the evaluation code, "eval_image_quality.py", why line 61 is commented？What is the real prompt that is fed to the model?

1701225586060

Thanks in advance!

teowu commented 12 months ago

The prompt starting "The image quality is" is not used here. After Q-Instruct, the result will only be slightly different either with or without the prompt starting.

You may try on both ways (or other prompt startings). 😏

Qiulin-W commented 12 months ago

The prompt starting "The image quality is" is not used here. After Q-Instruct, the result will only be slightly different either with or without the prompt starting.

You may try on both ways (or other prompt startings). 😏

Any insight here? Why the prompt can be discarded?

teowu commented 12 months ago

The prompt (i.e. User Input) is actually the qs, i.e. args.query. The "prompt starting" is like a condition (something like, telling MLLM that: you already output "The quality of the image is", then predict what the next word is). This is pretty useful for un-tuned MLLMs but not so effective with Q-Instruct (MLLMs are already prone to give simple single word outputs).