Fixed SimVP+Vit and SimVP+Swin

chengtan9907 / OpenSTL

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

https://openstl.readthedocs.io/en/latest/

Apache License 2.0

776 stars 129 forks source link

Fixed SimVP+Vit and SimVP+Swin #116

Closed mauriciodev closed 7 months ago

mauriciodev commented 8 months ago

The changes to TIMM introduced a bug that would prevent SimVP with Swin or Vit. The bug could be reproduced with the following:

python tools/train.py -d mmnist --lr 1e-3 -c configs/mmnist/simvp/SimVP_Swin.py --ex_name mmnist_simvp_swin

python tools/train.py -d mmnist --lr 1e-3 -c configs/mmnist/simvp/SimVP_Vit.py --ex_name mmnist_simvp_vit

The culprit was that the TIMM methods signatures for these blocks have two drop rates: attn and proj. I mapped both from the drop provided by openstl.

15650326960 commented 2 months ago

I also made a similar mistake, which is Proj's mistake. Have you solved the problem? 屏幕截图 2024-09-11 104416