search

THUDM / ImageReward

[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation
Apache License 2.0
1.18k stars 65 forks source link

Human Annotation Analysis #99

Open 292725651 opened 1 month ago

292725651 commented 1 month ago

The annotations with prompt, classification, overall_rating, image_text_alignment_rating, fidelity_rating are released. Will the annotations of image issues (i.e. repeated generation, body problems, fuzzy images, toxic, pornographic, violent, or violating protected groups in your papers) be released? Thank you! ![Uploading image.png…]()

xujz18 commented 1 month ago

Thanks for your attention! Please refer to #75 to find other annotation results.

292725651 commented 1 month ago

Thanks for your reply. However, I found that 'problem_data.json' contains only prompts. Does it contains the detail classifaction of problem (e.g. 'xxxxxx.webp' has body problems, 'xxxxxx.webp' has pornographic problems)?

crisnvg commented 1 month ago

+1, I also couldn't find overall_rating, image_text_alignment_rating, fidelity_rating