Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

Created by He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas J. Guibas from Stanford University, Google Inc., Princeton University, Facebook AI Research.

NOCS Teaser

Citation

If you find our work useful in your research, please consider citing:

 @InProceedings{Wang_2019_CVPR,
               author = {Wang, He and Sridhar, Srinath and Huang, Jingwei and Valentin, Julien and Song, Shuran and Guibas, Leonidas J.},
               title = {Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation},
               booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
               month = {June},
               year = {2019}
 }

Introduction

This is a keras and tensorflow implementation of Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation, a CVPR 2019 oral paper.

The repository includes:

Source code of NOCS.
Training code
Detection and evaluation code
Pre-trained weights
Example code for using Blender to render NOCS map

For more information, please visit the project page.

Requirements

This code has been tested with

CUDA 10.0 & cuDNN 7.41
Python 3.5
Tensorflow 1.14.0
Keras 2.3.0

Datasets

CAMERA Dataset: Training/Test/IKEA_backgrounds/Composed_depths
Composed depth images contain the depths of both foreground synthetic objects and background real scenes for all train and validation data
Real Dataset: Training/Test
Ground truth pose annotation (for an easier evaluation): Val&Real_test
Object Meshes

You can download the files and store them under data/.

NOTE: You are required to cite our paper if you use the dataset. The data is only for non-commercial use. Please reach out to us for other use cases.

Pretrain weights

You can find the following checkpoints in this download link:

NOCS RCNN jointly trained on CAMERA, Real & MS COCO with 32 bin classification setting
NOCS RCNN jointly trained on CAMERA, Real & MS COCO with regression setting
Mask RCNN pretrained on MS COCO dataset

You can download the checkpoints and store them under logs/.

Training

# Train a new model from pretrained COCO weight
python3 train.py

Detection and evaluation

# Detect using a checkpoint
python3 detect_eval.py --mode detect --ckpt_path=/logs/ckpt --draw

# Evaluate a checkpoint
python3 detect_eval.py --mode eval --ckpt_path=/output/ckpt

Rendering NOCS Map using blender 2.79b

# Render the NOCS map for the default cube in Blender
# See images/nocs_map_cube.png for the reference.
blender -b --python nocs_map_cube.py

hughw19 / NOCS_CVPR2019

readme