search

Karine-Huang / T2I-CompBench

[Neurips 2023] T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation
https://arxiv.org/pdf/2307.06350.pdf
MIT License
168 stars 5 forks source link

Question about human-eval dataset #6

Open vishaal27 opened 11 months ago

vishaal27 commented 11 months ago

Hey,

Thanks for the great work---I thoroughly enjoyed reading the paper, and congrats on the extensive evaluation of existing T2I models wrt compositionality. I am particularly interested in the results of your human study. As I understand it, you collected human ratings for each of 1800 unique images across 6 models and 150 prompts right (as mentioned in appendix section C.2)? Would you please be able to share the human rating data (both images and scores) publicly, it would be very useful!

Thanks!

Karine-Huang commented 7 months ago

Hi! Thank you for your interest in our work! We do have plans to release the human rating data, including the image-score pairs. Stay tuned for updates on our release schedule, and we hope it proves valuable for your research as well.

cyy-1234 commented 7 months ago

I am also looking forward to your release. Thank you.

vishaal27 commented 6 months ago

Hi @Karine-Huang, just checking back to see if there is any update on releasing the human rating data? Thanks! :)

Karine-Huang commented 6 months ago

Thank you for reaching out! I appreciate your follow-up. Currently, we are preparing for the human rating data release. We will make sure to keep you posted on any updates regarding the release.

vishaal27 commented 2 months ago

Hey @Karine-Huang, just checking to see if there were any updates on the human rating data release? Thanks!

Karine-Huang commented 2 weeks ago

Hello! Thank you for your patience. We have now released the human evaluation images. You can find them here.