Effect of the input size on the performance of SimVP

A4Bio / SimVP

The official implementation of the CVPR'22 paper SimVP: Simpler Yet Better Video Prediction.

215 stars 40 forks source link

Effect of the input size on the performance of SimVP #8

Open Hugh-Cai opened 1 year ago

Hugh-Cai commented 1 year ago

Hi, Gao I find the input size is almost smaller than 128 in the paper. So, I take a training on my own dataset which are same expect for different input size (128, and 512), leading to different performance:

for size 128, the performance has improved signigicantly;
but for size 512, the performance is always oscillatory during training.

What causes this situation? Can you help me with the questions or give me some suggestions?

Thanks

gaozhangyang commented 1 year ago

Thank you for raising a more challenging task. Currently, I don't have a better solution. But it is worth further research.

JunyaoHu commented 1 year ago

I trained on resolution 64 for KTH dataset and have the same problem. The possible reason is that training is difficult if the resolution is too high; If the resolution is too small, it is difficult to extract more details; Perhaps 128 is a more suitable compromise. More research is needed.