evalcrafter / EvalCrafter

[CVPR 2024] EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
http://evalcrafter.github.io
145 stars 7 forks source link

Representative results? #19

Open VesVlad opened 1 month ago

VesVlad commented 1 month ago

Good afternoon! Great job, thank you very much for posting the markup! Please tell me how linear regressions were trained for the final aggregation? I looked at the distribution of classes 0-5 in the markup and noticed that there was an imbalance and we would always converge to the average values ​​of 2-3. I hope I'm wrong, but could you please share the regression training?

Yaofang-Liu commented 1 month ago

Hi, we also found that if we train linear regression on all the label data, the result is not good since the quality of some labels are not that good. You may try this filtered human labels that we used in our paper
user_study_filtered.csv