search

Doubiiu / DynamiCrafter

[ECCV 2024] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors
Apache License 2.0
2.09k stars 165 forks source link

Evaluation on MSR-VTT test set #10

Open hiteshK03 opened 5 months ago

hiteshK03 commented 5 months ago

Hi, following on the above discussion, can you tell how you selected the 2048 samples for both the datasets? Because on calculating FVD for the entire dataset of MSR-VTT i.e. on 2990 videos, I got a score of 328 which is more than the reported value. Therefore, I was curious to know, if I am doing something wrong here.

Thanks.

Originally posted by @hiteshK03 in https://github.com/Doubiiu/DynamiCrafter/issues/6#issuecomment-1893053414

Doubiiu commented 5 months ago

Hi. Sorry for the late reply. I generated 2048 samples (use frame_stride=3) using the 1st frame of 2048 randomly selected videos in MSR-VTT. When computing FVD, please also use frame_stride=3 for the sampled real videos. Please contact me if you have any questions.

hiteshK03 commented 5 months ago

Hi, yeah I tried with the above configuration using frame_stride=3, but still got FVD value to be more than 300. Can you also share more about how you calculated FVD, so I can use the same to cross-check. Thanks.