How to predict next frames using one frame

wilson1yan / VideoGPT

MIT License

962 stars 115 forks source link

How to predict next frames using one frame #40

Open Mu-Yanchen opened 10 months ago

Mu-Yanchen commented 10 months ago

Hello! Thanks for your work, I have a question about prediction, in VideoGPT (BAIR), how should I predict 15 frames from 1 frame?

I don't know whether I need to use the sample method(def sample(self, n, batch=None)) to complete the prediction. I see in the sample method that we need to pass the “batch” dictionary, which has "video" and "label" keys. May I ask how to construct the batch dictionary parameter for a frame

guyyariv commented 9 months ago

Hey, any answer?

alexlioralexli commented 5 months ago

It seems like it's not possible, since the VQ-GAN downsamples in the time dimension.