search

A4Bio / SimVP

The official implementation of the CVPR'22 paper SimVP: Simpler Yet Better Video Prediction.
205 stars 40 forks source link

How to set the forecasting sequence length T ′ in your code #4

Closed fangzuliang closed 1 year ago

fangzuliang commented 2 years ago

image image

In your master code, the forecasting sequence length must be same as input sequence length T. but in your article, those two T could be different. Could you please implement the function setting dynamic forecasting sequence length?

gaozhangyang commented 1 year ago

The length of the model output and the length of the input are always the same. I think you are concerned with predicting T' output frames and T' $\neq$ T. If T'<T, you choose the top T' output as predictions. Otherwise, you can use the previously predicted frames as input for the next prediction until you get T' predicted frames. For example, if you need to predict 20 frames while the input supports 10 frames, you can input X (t=-9,-8,...,0) to predict Y1 (t=1,2,...,10), then enter Y1 (t=1,2,...,10) to predict Y2 (t=11,12,...,20). Finally, you can concatenate Y1 and Y2 as the prediction Y.

LeoXinhaoLee commented 1 year ago

Hi, thank you very much for releasing code for this inspiring work. Regarding the prediction length, say we input 10 previous frames and output 10 future frames, if our final goal is to predict 1 future frame, do we calculate and backpropagate the loss on the other 9 future frames during training or just on the first future frame?

Thank you very much for your time and help!