leo-p / papers

Papers and their summary (in issue)
22 stars 4 forks source link

Mask R-CNN #20

Open leo-p opened 7 years ago

leo-p commented 7 years ago

https://arxiv.org/pdf/1703.06870.pdf

We present a conceptually simple, flexible, and general framework for object instance segmentation. Our approach efficiently detects objects in an image while simultaneously generating a high-quality segmentation mask for each in- stance. The method, called Mask R-CNN, extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recogni- tion. Mask R-CNN is simple to train and adds only a small overhead to Faster R-CNN, running at 5 fps. Moreover, Mask R-CNN is easy to generalize to other tasks, e.g., al- lowing us to estimate human poses in the same framework. We show top results in all three tracks of the COCO suite of challenges, including instance segmentation, bounding-box object detection, and person keypoint detection. Without tricks, Mask R-CNN outperforms all existing, single-model entries on every task, including the COCO 2016 challenge winners. We hope our simple and effective approach will serve as a solid baseline and help ease future research in instance-level recognition. Code will be made available.

leo-p commented 7 years ago

Summary:

Inner workings:

The core operator of Faster R-CNN is the RoIPool which performs coarse spatial quantization for feature extraction but introduce misalignment for pixel-pixel comparison which is what segmentation is. The paper introduce a new layer RoIAlign that faithfully preserves exact spatial location.

One important point is that mask and class prediction are decoupled, the segmentation is proposed for each class without competing and the class predictor finally elects the winner.

Architecture:

Based on Faster R-CNN but with an added mask subnetwork that computes a segmentation mask for each class.

Different feature extractors and proposers are tried, see two examples below:

screen shot 2017-05-22 at 7 25 04 pm

Results:

Runs at about 200ms per frame on a GPU for segmentation (2 days training on a single 8-GPU) and 5 fps for pose estimation. Very impressive segmentation and pose estimation:

screen shot 2017-05-22 at 7 26 57 pm 1 screen shot 2017-05-22 at 7 29 26 pm