OpenGVLab / UniFormerV2

[ICCV2023] UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
https://arxiv.org/abs/2211.09552
Apache License 2.0
291 stars 18 forks source link

Integrate with huggingface #68

Open not-lain opened 7 months ago

not-lain commented 7 months ago

this pr will mainly add 3 methods to the uniformerv2 models

allowing your models to be easily integrated with huggingface using the PyTorchModelHubMixin class, I made this notebook https://colab.research.google.com/drive/1_H5LR5EclcBAhtyrarB_c49-TEceKoZU?usp=sharing explaining how to use it by the end of the notebook all users can load your model simply by

!pip install -q git+https://github.com/not-lain/UniFormerV2.git@integrate-with-huggingface # or your main branch when this is merged
from slowfast.models.uniformerv2_model import VisionTransformer
new_model = VisionTransformer.from_pretrained("not-lain/uniformerv2_b16") 
# no more renitializing the model and manually downloading the weights

I also made a huggingface space further showcasing how to use this pr.

Why you should integrate your model with huggingface ?

do not hesitate if you have any reviews on the pr or any questions.

Kind regards, Hafedh Hichri