LION: Latent Point Diffusion Models for 3D Shape Generation
NeurIPS 2022
utils/render_mitsuba_pc.py
Dependencies:
Setup the environment Install from conda file
conda env create --name lion_env --file=env.yaml
conda activate lion_env
# Install some other packages
pip install git+https://github.com/openai/CLIP.git
# build some packages first (optional)
python build_pkg.py
Tested with conda version 22.9.0
Using Docker
bash ./docker/build_docker.sh
bash ./docker/run.sh
run python demo.py
, will load the released text2shape model on hugging face and generate a chair point cloud. (Note: the checkpoint is not released yet, the files loaded in the demo.py
file is not available at this point)
python ./script/check_sum.py ./lion_ckpt.zip
./lion_ckpt/
./data/ShapeNetCore.v2.PC15k
or edit the pointflow
entry in ./datasets/data_path.py
for the ShapeNet dataset path. bash ./script/train_vae.sh $NGPU
(the released checkpoint is trained with NGPU=4
on A100) .comet_api
file under the current folder, write the api key as {"api_key": "${COMET_API_KEY}"}
in the .comet_api
filebash ./script/train_prior.sh $NGPU
(the released checkpoint is trained with NGPU=8
with 2 node on V100)./data/shapenet_render/
or edit the clip_forge_image
entry in ./datasets/data_path.py
./datasets/pointflow_datasets.py
with the render_img_path
, you may need to cutomize this variable depending of the folder structure bash ./script/train_prior_clip.sh $NGPU
.comet_api
under this LION
folder, example of the .comet_api
file:
{"api_key": "...", "project_name": "lion", "workspace": "..."}
.wandb_api
file, and set the env variable export USE_WB=1
before training
{"project": "...", "entity": "..."}
export USE_TFB=1
before trainingutils/utils.py
files for the details of the experiment logger; I usually use comet-ml for my experiments./datasets/test_data/
checkpoint="./lion_ckpt/unconditional/airplane/checkpoints/model.pt"
bash ./script/eval.sh $checkpoint # will take 1-2 hour
./datasets/test_data/
python ./script/compute_score.py
(Note: for ShapeNet-Vol data and table 21, 20, need to set norm_box=True
)@inproceedings{zeng2022lion,
title={LION: Latent Point Diffusion Models for 3D Shape Generation},
author={Xiaohui Zeng and Arash Vahdat and Francis Williams and Zan Gojcic and Or Litany and Sanja Fidler and Karsten Kreis},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2022}
}