Official repository of "BodyShapeGPT: SMPL Body Shape Manipulation with LLMs" [ECCVW 2024]
In European Conference on Computer Vision Workshops (ECCVW) 2024, Workshop on Foundation Models for 3D Humans
Generative AI models provide a wide range of tools capable of performing complex tasks in a fraction of the time it would take a human. Among these, Large Language Models (LLMs) stand out for their ability to generate diverse texts, from literary narratives to specialized responses in different fields of knowledge. This paper explores the use of fine-tuned LLMs to identify physical descriptions of people, and subsequently create accurate representations of avatars using the SMPL-X model by inferring shape parameters. Our results demonstrate that LLMs can be trained to understand and manipulate the shape space of SMPL, allowing the control of 3D human shapes through natural language. This approach promises to improve human-machine interaction and opens new avenues for customization and simulation in virtual environments.
Please follow the steps below to use BodyShapeGPT.
git clone https://github.com/baldoarbol/BodyShapeGPT
pip install -r requirements.txt
Use this OneDrive link to download the LoRa model weights (.safetensors file). After downloading, place the .safetensors file into the "weights" folder located within the project directory.
You can use the script demo.py to try our model. Usage:
python3.9 demo.py "[avatar description]"
demo.py runs a single inference on the provided prompt and outputs the result betas for a gender-neutral SMPL-X body. Feel free to customize the script as needed to better fit your use case.
In the root directory of this project, you'll find the BodyShapeGPT_dataset.jsonl
file, which contains 21,000 natural language descriptions of avatars, each paired with their corresponding gender-neutral SMPL-X shape parameters.
If you find this repository useful please cite our work:
@InProceedings{R-ARBOL_2024_ECCV_BodyShapeGpt,
author = {R. Árbol, Baldomero and Casas, Dan},
title = {BodyShapeGPT: SMPL Body Shape Manipulation with LLMs},
booktitle = {European Conference on Computer Vision Workshops (ECCVW) 2024},
year = {2024}
}