Evaluation of text and image output. We first employ an answer ranking strategy to select the most likely text prediction. If it matches the ground truth, we evaluate the image output using the CLIP similarity score [50] between the generated image and each candidate. The model is deemed correct only if both text and image predictions match the ground truth.
I'm a little confused. How do I generate the image, first spell the corresponding text answer and then generate the image, or generate the image directly from the question?
Thanks for your work!
Hello. I saw this description in your paper.
I'm a little confused. How do I generate the image, first spell the corresponding text answer and then generate the image, or generate the image directly from the question? Thanks for your work!