DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction (3DV 2021)

[Paper] [Introduction Video (2 minutes)] [Introduction Slides] [Full Video (10 minutes)] [Full Slides] [Poster]

This repo is the implementation of DPC. Created by Itai Lang*, Dvir Ginzburg*, Shai Avidan, and Dan Raviv from Tel Aviv University.
*Equal contribution

Architecture Cross Similarity

Tested environment

Python 3.6
PyTorch 1.6
CUDA 10.2

Lower CUDA and PyTorch versions should work as well.

Installation
Datasets
Train
Inference
Cite

Installation

Please follow installation.sh or simply run

bash installation.sh

Datasets

The method was evaluated on:

SURREAL
- 230k shapes (DPC uses the first 2k).
- Dataset website
- This code downloads and preprocesses SURREAL automatically.
SHREC’19
- 44 Human scans.
- Dataset website
- This code downloads and preprocesses SURREAL automatically.
SMAL
- 10000 animal models (2000 models per animal, 5 animals).
- Dataset website
- Due to licencing concerns, you should register to SMAL and download the dataset.
- You should follow data/generate_smal.md after downloading the dataset.
- To ease the usage of this benchmark, the processed dataset can be downloaded from here. Please extract and put under data/datasets/smal
TOSCA
- 41 Animal figures.
- Dataset website
- This code downloads and preprocesses TOSCA automatically.
- To ease the usage of this benchmark, the processed dataset can be downloaded from here. Please extract and put under data/datasets/tosca

Training

For training run

python train_point_corr.py --dataset_name <surreal/tosca/shrec/smal>

The code is based on PyTorch-Lightning, all PL hyperparameters are supported. (limit_train/val/test_batches, check_val_every_n_epoch etc.)

Tensorboard support

All metrics are being logged automatically and stored in

output/shape_corr/DeepPointCorr/arch_DeepPointCorr/dataset_name_<name>/run_<num>

Run tesnroboard --logdir=<path> to see the the logs.

Example of tensorboard output:

tensorboard

Inference

For testing, simply add --do_train false flag, followed by --resume_from_checkpoint with the relevant checkpoint.

python train_point_corr.py --do_train false  --resume_from_checkpoint <path>

Test phase visualizes each sample, for faster inference pass --show_vis false.

We provide a trained checkpoint repreducing the results provided in the paper, to test and visualize the model run

python train_point_corr.py --show_vis --do_train false --resume_from_checkpoint data/ckpts/surreal_ckpt.ckpt

Results

Citing & Authors

If you find this repository helpful feel free to cite our publication -

@InProceedings{lang2021dpc,
  author = {Lang, Itai and Ginzburg, Dvir and Avidan, Shai and Raviv, Dan},
  title = {{DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction}},
  booktitle = {Proceedings of the International Conference on 3D Vision (3DV)},
  pages = {1442--1451},
  year = {2021}
}

Contact: Dvir Ginzburg, Itai Lang.

dvirginz / DPC

readme