A deep learning library for video understanding research.

Check the website for more information.

A PyTorchVideo-accelerated X3D model running on a Samsung Galaxy S10 phone. The model runs ~8x faster than real time, requiring roughly 130 ms to process one second of video.	A PyTorchVideo-based SlowFast model performing video action detection.

X3D model Web Demo

Integrated to Huggingface Spaces with Gradio. See demo:

Introduction

PyTorchVideo is a deeplearning library with a focus on video understanding work. PytorchVideo provides reusable, modular and efficient components needed to accelerate the video understanding research. PyTorchVideo is developed using PyTorch and supports different deeplearning video components like video models, video datasets, and video-specific transforms.

Key features include:

• Based on PyTorch: Built using PyTorch. Makes it easy to use all of the PyTorch-ecosystem components.
• Reproducible Model Zoo: Variety of state of the art pretrained video models and their associated benchmarks that are ready to use. Complementing the model zoo, PyTorchVideo comes with extensive data loaders supporting different datasets.
• Efficient Video Components: Video-focused fast and efficient components that are easy to use. Supports accelerated inference on hardware.

Updates

• Aug 2021: Multiscale Vision Transformers has been released in PyTorchVideo, details can be found from here.

Installation

Install PyTorchVideo inside a conda environment(Python >=3.7) with

pip install pytorchvideo

For detailed instructions please refer to INSTALL.md.

License

PyTorchVideo is released under the Apache 2.0 License.

Tutorials

Get started with PyTorchVideo by trying out one of our tutorials or by running examples in the tutorials folder.

Model Zoo and Baselines

We provide a large set of baseline results and trained models available for download in the PyTorchVideo Model Zoo.

Contributors

Here is the growing list of PyTorchVideo contributors in alphabetical order (let us know if you would like to be added): Aaron Adcock, Amy Bearman, Bernard Nguyen, Bo Xiong, Chengyuan Yan, Christoph Feichtenhofer, Dave Schnizlein, Haoqi Fan, Heng Wang, Jackson Hamburger, Jitendra Malik, Kalyan Vasudev Alwala, Matt Feiszli, Nikhila Ravi, Ross Girshick, Tullie Murrell, Wan-Yen Lo, Weiyao Wang, Xiaowen Lin, Yanghao Li, Yilei Li, Zhengxing Chen, Zhicheng Yan.

Development

We welcome new contributions to PyTorchVideo and we will be actively maintaining this library! Please refer to CONTRIBUTING.md for full instructions on how to run the code, tests and linter, and submit your pull requests.

Citing PyTorchVideo

If you find PyTorchVideo useful in your work, please use the following BibTeX entry for citation.

@inproceedings{fan2021pytorchvideo,
    author =       {Haoqi Fan and Tullie Murrell and Heng Wang and Kalyan Vasudev Alwala and Yanghao Li and Yilei Li and Bo Xiong and Nikhila Ravi and Meng Li and Haichuan Yang and  Jitendra Malik and Ross Girshick and Matt Feiszli and Aaron Adcock and Wan-Yen Lo and Christoph Feichtenhofer},
    title = {{PyTorchVideo}: A Deep Learning Library for Video Understanding},
    booktitle = {Proceedings of the 29th ACM International Conference on Multimedia},
    year = {2021},
    note = {\url{https://pytorchvideo.org/}},
}