songweige / content-debiased-fvd

[CVPR 2024] On the Content Bias in Fréchet Video Distance
https://content-debiased-fvd.github.io/
MIT License
89 stars 6 forks source link

Expected result for example in readme #2

Closed roudimit closed 3 months ago

roudimit commented 3 months ago

Hi, thanks for the nice repo! What is the expected result when running the example in the readme? I got the following numbers, are they correct?

from cdfvd import fvd
evaluator = fvd.cdfvd('videomae', ckpt_path=None)
evaluator.load_videos('ucf101', data_type='stats_pkl', resolution=128, sequence_length=16)
evaluator.compute_fake_stats(evaluator.load_videos('./example_videos/', data_type='video_folder'))
print(evaluator.compute_fvd_from_stats())
>>>1064.6156430306792

I3D result using evaluator = fvd.cdfvd('i3d', ckpt_path=None)

>>>3383.9670431314958
songweige commented 3 months ago

Hi @roudimit, yes, these numbers look correct to me, up to numerical errors! These are what I got:

>>>>1064.615641522972

For I3D:

>>>>3383.964558389454
roudimit commented 3 months ago

Thanks!