evalcrafter / EvalCrafter

[CVPR 2024] EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
http://evalcrafter.github.io
115 stars 7 forks source link

Incorrect DOVER technical and aesthetics results order? #13

Open funnylittleman opened 2 months ago

funnylittleman commented 2 months ago

Results are indexed incorrectly in fuse_resultsfunction. They are ordered as ['technical', 'aesthetic'], so tshould be computed with results[0] and awith results[1].

https://github.com/evalcrafter/EvalCrafter/blob/d3ff35c1a31e65fb365209c945c6e0848837a3c7/metrics/DOVER/evaluate_a_set_of_videos.py#L36