STAF: 3D Human Mesh Recovery from Video with Spatio-Temporal Alignment Fusion

Getting Started

Installation & Clone the repo [Environment on Linux (Ubuntu 18.04 with python >= 3.7)]

# Install the requirements using `virtualenv`: 
cd $PWD/STAF
source scripts/install_pip.sh

Download the Required Data

You can download the required data and the pre-trained STAF model from here. You need to unzip the contents and the data directory structure should follow the below hierarchy.

${ROOT}  
|-- data  
|   |-- base_data    

Running the Demo

We have prepared a demo code to run STAF on arbitrary videos. To do this you can just run:

python demo.py --vid_file demo_video.mp4 --gpu 0

Training and Evaluation

Please refer to training issue and evaluation issue.

Acknowledgments

Part of the code is borrowed from the following projects, including PyMAF, MPS-Net. Many thanks to their contributions. Special thanks to camenduru for Colab Demo!

Citation

If you find this repository useful, please consider citing our paper and lightning the star:

@ARTICLE{yao2024staf,
  author={Yao, Wei and Zhang, Hongwen and Sun, Yunlian and Tang, Jinhui},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={STAF: 3D Human Mesh Recovery from Video with Spatio-Temporal Alignment Fusion}, 
  year={2024},
  volume={},
  number={},
  pages={1-1},
  keywords={Hidden Markov models;Three-dimensional displays;Feature extraction;Image reconstruction;Solid modeling;Biological system modeling;Coherence;3D human mesh recovery;temporal coherence;feature pyramid;attention model},
  doi={10.1109/TCSVT.2024.3410400}}