Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi, Visual Geometry Group, University of Oxford. In CVPR 2020 (Best Paper Award).
We propose a method to learn weakly symmetric deformable 3D object categories from raw single-view images, without ground-truth 3D, multiple views, 2D/3D keypoints, prior shape models or any other supervision.
author: samuel ko
This repo is target to replace the neural_renderer from CVPR 2018 with the sota renderer ICCV 2019 Soft Rasterizer provided by pytorch3d.
In order to compatible with the pytorch3d API, we change a little bit in unsup3d/renderer/renderer.py
& unsup3d/renderer/utils.py
.
However, the effect of this implemention is not good. I will still work on this and try to fix that problem.
SoftRas(trained on celebA)
conda env create -f environment.yml
OR manually:
conda install -c conda-forge scikit-image matplotlib opencv moviepy pyyaml tensorboardX
conda install pytorch==1.4.0 torchvision cudatoolkit=10.0 -c pytorch
Note: The code is tested with PyTorch 1.4.0 and CUDA 10.0 on Ubuntu 18.04. A GPU version is required for training and testing, since the neural_renderer package only has GPU implementation.
conda install --channel https://conda.anaconda.org/pytorch3d pytorch3d
This package is optional for the demo. It allows automatic human face detection.
pip install facenet-pytorch
img_celeba.7z
) from their website and run celeba_crop.py
in data/
to crop the images.download_synface.sh
provided in data/
.download_cat.sh
provided in data/
.Please remember to cite the corresponding papers if you use these datasets.
python -m demo.demo --input demo/images/human_face --result demo/results/human_face --checkpoint pretrained/pretrained_celeba/checkpoint030.pth
Check the configuration files in experiments/
and run experiments, eg:
python run.py --config experiments/train_gdh.yml --gpu 0 --num_workers 2
@InProceedings{Wu_2020_CVPR,
author = {Shangzhe Wu and Christian Rupprecht and Andrea Vedaldi},
title = {Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild},
booktitle = {CVPR},
year = {2020}
}