search

XiYe20 / VPTR

The repository for paper VPTR: Efficient Transformers for Video Prediction
MIT License
88 stars 19 forks source link

pretrained models #3

Closed tusiyi closed 1 year ago

tusiyi commented 1 year ago

Hello! I have a problem about loading the pretrained you provide when I want to test VPTR using KTH dataset, and I shows the screenshot below. It seems that the state_dict of KTH_FAR.tar doesn't match with the model defined in the test_VPTR, for example, the model requires 'transformer.encoder.layers.0.SLMHSA.attn.k_proj....' , but the state_dict only have in_proj. Should I change VPTRFormerFAR instantiation when testing KTH dataset? (using MovingMNIST didn't have this problem). I show the debug process, the state_dict is in red in the debugger, and the model VPTRFormerFAR structure is shown on the right. pretrained-KTH-FAR-not match key

XiYe20 commented 1 year ago

Hi, Thank you very much for the information. I uploaded the wrong checkpoint for the KTH_FAR model. I have uploaded the right one: KTH_FAR_new.tar, you can download it and have a try. For testing, set the model_flag to be "FAR", also set the "out_layer" of VPTRDec to be "Tanh" for KTH/BAIR dataset.

tusiyi commented 1 year ago

Thanks a lot!