search

lucidrains / TimeSformer-pytorch

Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification
MIT License
686 stars 86 forks source link

Imagenet Pretrained Weights #9

Open RaivoKoot opened 3 years ago

RaivoKoot commented 3 years ago

Thanks for the work! In their paper they say For all our experiments, we adopt the “Base” ViT model architecture (Dosovitskiy et al., 2020) pretrained on ImageNet.

I know that you said the official weights trained on kinetics and such are not officially released yet. However, I am not interested in those but am actually in need of the initial weights of the network just based on ViT Imagenet pretraining. I need to train this implementation of yours starting from those. From what it looks like, you don't have weights for this implementation that come from imagenet pretraining, do you?

jwohlwend commented 3 years ago

+1

MohamedOsman1998 commented 3 years ago

+1

tcapelle commented 3 years ago

I am not sure it is possible, but ViT weights are available on timm and others.

RaivoKoot commented 3 years ago

For now, I found this repository that includes ViT initialization (https://github.com/m-bain/video-transformers).

tcapelle commented 3 years ago

You can even download a pretrained ViT from huggingface: https://huggingface.co/google/vit-large-patch16-224