search

Vchitect / Latte

Latte: Latent Diffusion Transformer for Video Generation.
Apache License 2.0
1.45k stars 147 forks source link

FVD values of PVDM are strange #6

Open sihyun-yu opened 5 months ago

sihyun-yu commented 5 months ago

Hi, I am the first author of PVDM, and I just checked the FVD values of PVDM are much worse than the values that I reported in the paper. Could you tell me why such differences exist?

Many people tried (and succeeded) to reproduce the values, so it is weird to me.

maxin-cn commented 5 months ago

Hi, I am the first author of PVDM, and I just checked the FVD values of PVDM are much worse than the values that I reported in the paper. Could you tell me why such differences exist?

Many people tried (and succeeded) to reproduce the values, so it is weird to me.

Hello, thanks for your interest of our work. For UCF101 and Skytimelapse datasets, we followed the paper and used provided pre-trained checkpoints for evaluation. However, we were unable to reproduce the reported results. It would be very helpful if you could provide any details or a complete evaluation code on how to obtain the results.

sihyun-yu commented 5 months ago

Just a quick check:

Because with this setup, I got many mails that the results can be reproduced with the checkpoints.

sihyun-yu commented 4 months ago

Hi,

there even exists a recent arxiv paper that has succeeded to reproduce the values of PVDM: https://arxiv.org/abs/2402.13729v1 If you face difficulties to reproduce the values, you should ask authors before making it public..

sihyun-yu commented 3 months ago

Can you please reply to this issue?

maxin-cn commented 3 months ago

Can you please reply to this issue?

Thank you very much for sharing the info and sorry for missing your question. We will thoroughly review the results with the information you've provided. We would greatly appreciate any additional details you could offer regarding the reimplementation of your work. Many thanks again.