search

asyml / vision-transformer-pytorch

Pytorch version of Vision Transformer (ViT) with pretrained models. This is part of CASL (https://casl-project.github.io/) and ASYML project.
https://asyml.io/
Apache License 2.0
342 stars 58 forks source link

您好，请问怎么把这个用于视频分类？比如5分钟的短视频 #6

Open dotsonliu opened 3 years ago

christy-yuan-li commented 3 years ago

Thanks for your question. We haven't applied the model to video classification. However, you could use the ViT as a base model to encode each frame of your video.

dotsonliu commented 3 years ago

when there has hundreds of frames ,How to deal with it?

christy-yuan-li commented 3 years ago

Thank you for your question. The problem of how to efficiently process videos is interesting, but not the focus of this repo. We are happy to discuss this potential application with you, but maybe at some other venue. I would suggest you to check related literature first. My previous response was mainly to convey the idea that ViT can be used for processing images in general.

runningJ commented 3 years ago

Directly using VIT to process video may result in bad results.