search

prosperolo / GST

Official implementation of "GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers"
https://abdullahamdi.com/gst/
BSD 3-Clause "New" or "Revised" License
66 stars 7 forks source link

readme

GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers

Installation

  1. Create a conda environment 

    conda create --name gst python=3.10
conda activate gst

  2. Install Pytorch following the official instructions. Python/Pytorch combination that was verified to work is: Python 3.10, Pytorch 2.4.1, CUDA 12.4 (Ubuntu 22.04)

  3. Install 4DHumans following the official instructions. Download the SMPL model according to the repo README instructions, then run the demo.py script to download the pretrained models and store them under the correct path

  4. Install Gaussian splatting renderer diff-gaussian-rasterization

  5. Install simple-knn

  6. Install other requirements

    pip install -r requirements.txt

Data preparation

To train on the HuMMan dataset

  1. Download the dataset following the official instruction
  2. Unzip the data and change the HUMMAN_DATASET_ROOT to the unzipped folder inside scene/humman.py

Training

python train_network.py +dataset=humman

Evaluation

Download the pretrained model and config from Huggingface. Then run 

python eval.py path_to_model.pth path_to_config.yaml num_of_images_to_save

The first num_of_images_to_save examples will be saved inside the eval_out folder and will look like the image below (input image on the left, rendering in the top row, ground truth image in the bottom row)