uoip/SSD-variants - Githubissues

This is a learning project trying to implement some varants of SSD in pytorch. SSD is a one-stage object detector, probably "currently the best detector with respect to the speed-vs-accuracy trade-off". There are many follow-up papers that either further improve the detection accuracy, or incorporate techniques like image segmentation to be used for Scene Understanding(e.g. BlitzNet), or modify SSD to detect rotatable objects(e.g. DRBox), or apply SSD to 3d object detection(e.g. Frustum PointNets):

SSD - "SSD: Single Shot MultiBox Detector" (2016) arXiv:1512.02325 , github
DSSD - "DSSD : Deconvolutional Single Shot Detector" (2017) arXiv:1701.06659
RRC - "Accurate Single Stage Detector Using Recurrent Rolling Convolution" (2017) arXiv:1704.05776 , github
RUN - "Residual Features and Unified Prediction Network for Single Stage Detection" (2017) arXiv:1707.05031
DSOD - "DSOD: Learning Deeply Supervised Object Detectors from Scratch" (2017) arXiv:1708.01241 , github
BlitzNet - "BlitzNet: A Real-Time Deep Network for Scene Understanding" (2017) arXiv:1708.02813 , github
RefineDet - "Single-Shot Refinement Neural Network for Object Detection" (2017) arXiv:1711.06897 , github
DRBox - "Learning a Rotation Invariant Detector with Rotatable Bounding Box" (2017) arXiv:1711.09405 , github
Frustum PointNets - "Frustum PointNets for 3D Object Detection from RGB-D Data" (2017) arXiv:1711.08488

Overview

Model	publish time	Backbone	input size	Boxes	FPS	VOC07	VOC12	COCO
SSD300	2016	VGG-16	300 × 300	8732	46	77.2	75.9	25.1
SSD512	2016	VGG-16	512 × 512	24564	19	79.8	78.5	28.8
SSD321	2017.01	ResNet-101	321 × 321	17080	11.2	77.1	75.4	28.0
SSD513	2017.01	ResNet-101	513 × 513	43688	6.8	80.6	79.4	31.2
DSSD321	2017.01	ResNet-101	321 × 321	17080	9.5	78.6	76.3	28.0
DSSD513	2017.01	ResNet-101	513 × 513	43688	5.5	81.5	80.0	33.2
RUN300	2017.07	VGG-16	300 × 300	11640	64 (Pascal)	79.1	77.0
DSOD300	2017.08	DS/64-192-48-1	300 × 300		17.4	77.7	76.3	29.3
BlitzNet300	2017.08	ResNet-50	300 × 300	45390	24	78.5	75.4	29.7
BlitzNet512	2017.08	ResNet-50	512 × 512	32766	19.5	80.7	79.0	34.1
RefineDet320	2017.11	VGG-16	320 × 320	6375	40.3	80.0	78.1	29.4
RefineDet512	2017.11	VGG-16	512 × 512	16320	24.1	81.8	80.0	33.0
RefineDet320	2017.11	ResNet-101	320 × 320					32.0
RefineDet512	2017.11	ResNet-101	512 × 512					36.4

RRC	2017.04	VGG-16	1272 × 375
DRBox	2017.11	VGG-16	300 × 300
Frustum PointNets rgb part	2017.11	VGG-16	1280 × 384

FPS: # of processed images per second on Titan X GPU (batch size is 1)
VOC07: PASCAL 2007 detection results(mAP), training data: 07+12(07 trainval + 12 trainval)
VOC12: PASCAL 2012 detection results(mAP), training data: 07++12(07 trainval + 07 test + 12 trainval)
COCO: MS COCO 2015 test-dev detection results(mAP@[0.5:0.95]), train on trainval35k

All backbone networks above have been pre-trained on ImageNet CLS-LOC dataset, except DSOD, it's "training from scratch".

Implemented

[x] SSD
[x] RRC
[x] RUN
[x] DSOD
[x] BlitzNet (detection part)
[ ] DRBox
[ ] Frustum PointNets

Note: "Implemented" above means the code of the model is almost done, it doesn't mean I have trained it, or even reproduced the results of original paper. Actually, I have only trained SSD300 on VOC07, the best result I got is 76.5%, lower than 77.2% reported in SSD paper. I'll continue this project when I find out what's the problem.

Requirements

Python 3.6+
numpy
cv2
pytorch
tensorboardX

Dataset

Download dataset VOC2007 and VOC2012, put them under VOCdevkit directory:

VOCdevkit
-| VOC2007
   -| Annotations
   -| ImageSets
   -| JPEGImages
   -| SegmentationClass
   -| SegmentationObject
-| VOC2012
   -| Annotations
   -| ImageSets
   -| JPEGImages
   -| SegmentationClass
   -| SegmentationObject

Usage

train:

python train.py --cuda --voc_root path/to/your/VOCdevkit --backbone path/to/your/vgg16_reducedfc.pth
The backbone network vgg16_reducedfc.pth is from repo amdegroot/ssd.pytorch (download link: https://s3.amazonaws.com/amdegroot-models/vgg16_reducedfc.pth).

evaluate:

python train.py --cuda --test --voc_root path/to/your/VOCdevkit --checkpoint path/to/your/xxx.pth

show demo:

python train.py --cuda --demo --voc_root path/to/your/VOCdevkit --checkpoint path/to/your/xxx.pth

Results

VOC07 mAP

models	my result	paper result
SSD300	76.5%	77.2%

to be continued