junxnone / tio

Log

Other

10 stars 5 forks source link

YOLACT #710

Open junxnone opened 4 years ago

junxnone commented 4 years ago

YOLACT/YOLACT++

Arch

ResNet -> C1/C2/C3/C4/C5
C5 + Conv -> P5
C4 + Conv + P5 -> P4
C3 + Conv + P4 -> P3 -> Protonet
P5 + Conv -> P6
P6 + Conv -> P7

P3~P7 -> Prediction Head

Protonet

基于 FCN
使用 FPN 作为 backbone
输出 k channel mask

Prototype mask * coefficient = final mask

Head

Output : 4 + C + k = anchor + class + mask coefficient
Classes: c positive labels + 1 background label
OHEM: neg:pos = 3:1

Mask Assembly

P - hxwxk prototyep masks
C - nxk mask coefficients - NMS 和 Threshold 后剩 n instance
σ - sigmoid

Loss

classiﬁcation loss L-cls - 同 SSD
box regression loss L-box - 同 SSD
mask loss L-mask - assembled masks M & GT masks pixel-wisebinary cross entropy

Fast NMS

parallel remove detections
11.8ms faster 0.1mAP reducing vs traditional NMS
steps
- 计算 c x n x n IoU 矩阵
- c classes : c 个分类
- top n detections: 按 score 排序后的 n 个 BBox
- 计算每列中最大 IoU
- 移除 IoU 大于 t 的 BBox

Semantic Segmentation Loss - extra losses

最大 feature map + 1x1 conv layer + sigmoid = c channel mask
提升 0.4 mAP

Crop & Threshold

mask coefficient + prototype mask + crop + threshold -> final mask
crop : 取 BBox 内的 mask
Threshold : 根据 Confidence 取 mask

YOLACT++

Fast Mask Re-Scoring Network

Add Fast mask re-scoring branch
- Input : cropped mask
- 不同于 MS-RCNN
  - 没有取 ROI Align layer
  - 没有 FC Layer
Inference Time +1.2 ms Throughput -1.4 fps
- 如果使用 原版 MS RCNN MaskIoU Head, 则 +28 ms -16.9 fps

Deformable Convolution with Intervals

为了 Throughput , 不使用整个 Deformable Conv Module, 只替换 C3 ~ C5 中的 3x3 Conv Layer 为 3x3 Deformable Conv Layer - +1.8 mAP - +8 ms
Final DCN: interval = 3 - +2.8 ms +1.6 mAP
- 每隔3个卷积层，替换一个标准卷积为可变形卷积

Optimized Prediction Head

anchor scales & aspect ratios
- 实验一: aspect ratios [1,1/2,2] -> [1,1/2,2,1/3,3] - +5/3x anchors
- 实验二: multi-scales ([1x, 2^(1/3)x, 2^(2/3)x]) - +3x anchors

YOLACT++ Improvements

YOLACT/YOLACT++ Performance vs Others Network

mask mAP @COCO test-dev
Throughput @Titan Xp
- Input size = 550x550

存在的问题

定位失败
- 重叠
泄露
- 当 BBox 不准时会出现

Reference

junxnone commented 4 years ago

305 #524