evalcrafter / EvalCrafter

[CVPR 2024] EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
http://evalcrafter.github.io
113 stars 7 forks source link

a typo in the 321st prompt #2

Closed jidedaka closed 1 month ago

jidedaka commented 9 months ago

There seem to be a typo in the 321st prompt In 3d game style, Salvador Dalí with a robotic half face. Right?

jidedaka commented 9 months ago

And there seem to be a redundant"style" in the 285th prompt.

jidedaka commented 9 months ago

The 100th and 109th prompts appear to be the same.

jidedaka commented 9 months ago

The 350th prompt appears to have a type ofA

jidedaka commented 9 months ago

The 6th and 19th prompts appear to be the same.

jidedaka commented 9 months ago

The 257th and 265th prompts appear to have typo "ofTwo"

jidedaka commented 9 months ago

There are 17 "ofa" typos in the prompt list.

vinthony commented 9 months ago

hi, sorry for the inconvenient. It is the first version caused by some style template generation. Fixed.

jidedaka commented 9 months ago

Thank you, Cun. But it appears that you haven't fixed all the problems I listed completely. It would be great if you could check again.

Yaofang-Liu commented 4 months ago

Hi jidedaka, thank you very much for your careful checking. Please ref to the Appendix D. 7 about Benchmark Stability and Noise Variables analysis in our updated paper https://arxiv.org/html/2310.11440v3, which shows that these typos has subtle effects on our benchmark. For the sake of convenience, we kept this small amount of noise in our current version. Please feel free to ask any further questions. Many thanks!