search

YuliangXiu / PuzzleAvatar

[SIGGRAPH Asia 2024] PuzzleAvatar: Assembling 3D Avatars from Personal Albums
https://puzzleavatar.is.tue.mpg.de/
Other
140 stars 3 forks source link

readme

PuzzleAvatar:
Assembly of Avatar from Unconstrained Photo Collections

Yuliang Xiu · Yufei Ye · Zhen Liu · Dimitris Tzionas · Michael J. Black

SIGGRAPH Asia 2024 (Journal Track)


Paper PDF Project Page youtube views

PuzzleAvatar reconstructs a textured 3D clothed human from **unconstrained photo collections** | Works For | Without Requiring | |:---:|:---:| | ✅ Any number of photos | ❌ Human body pose (3D HPS, 2D Keypoints, etc.) | | ✅ In any poses | ❌ Camera pose| | ✅ From any views | ❌ Geometric cues (depth, normal, etc.) | | ✅ With any cropping or occlusion | ❌ Re-projection losses |


News :triangular_flag_on_post:


Getting Started

  1. Set up the paths in scripts/env.sh.
  2. Please follow the Installation Instruction to setup all the required packages.
  3. Run PuzzleAvatar (Grounded-SAM $\rightarrow$ PuzzleBooth $\rightarrow$ SDS, takes about 4 hours) 
    bash scripts/run.sh data/human/yuliang results/human/yuliang human_yuliang

    The results will be saved in the experiment folder results/human/yuliang

Dataset (Coming soon...)

Acknowledgment

This implementation is built based on TeCH, Break-A-Scene, Grounded SAM, GPT-4V(ision), Stable Diffusion 2-1-base, BOFT-DreamBooth, Stable Dreamfusion, ECON.

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No.860768 (CLIPE Project)

Kudos to all of our amazing contributors! PuzzleAvatar thrives through open-source. In that spirit, we welcome all kinds of contributions from the community.

Contributor avatars are randomly shuffled.

License

This code and model are available for non-commercial scientific research purposes as defined in the LICENSE file. By downloading and using the code and model you agree to the terms in the LICENSE.

Disclosure

MJB has received research gift funds from Adobe, Intel, Nvidia, Meta/Facebook, and Amazon. MJB has financial interests in Amazon and Meshcapade GmbH. While MJB is a co-founder and Chief Scientist at Meshcapade, his research in this project was performed solely at, and funded solely by, the Max Planck Society.

Contact

For technical questions, please contact yuliang.xiu@tue.mpg.de

For commercial licensing, please contact ps-licensing@tue.mpg.de

Citation

@article{xiu2024puzzleavatar,
  title={PuzzleAvatar: Assembling 3D Avatars from Personal Albums},
  author={Xiu, Yuliang and Ye, Yufei and Liu, Zhen and Tzionas, Dimitrios and Black, Michael J},
  journal={ACM Transactions on Graphics (TOG)},
  year={2024},
  publisher={ACM New York, NY, USA}
}

PuzzleAvatar is mainly built upon TeCH, please also kindly cite it

@inproceedings{huang2024tech,
  title={{TeCH: Text-guided Reconstruction of Lifelike Clothed Humans}},
  author={Huang, Yangyi and Yi, Hongwei and Xiu, Yuliang and Liao, Tingting and Tang, Jiaxiang and Cai, Deng and Thies, Justus},
  booktitle={International Conference on 3D Vision (3DV)},
  year={2024}
}