RetinaNet in Pytorch

Fast and accurate single stage object detection with end-to-end GPU optimization.

Based on the repo github.com/NVIDIA/retinanet-examples. All credit goes to the generous folks at NVIDIA.

Description

RetinaNet is a single shot object detector with multiple backbones offering various performance/accuracy trade-offs.

It is optimized for end-to-end GPU processing using:

• The PyTorch deep learning framework with ONNX support
• NVIDIA Apex for mixed precision and distributed training
• NVIDIA DALI for optimized data pre-processing
• NVIDIA TensorRT for high-performance inference
• NVIDIA DeepStream for optimized real-time video streams support

Disclaimer

This is a research project, not an official NVIDIA product.

Performance

The detection pipeline allows the user to select a specific backbone depending on the latency-accuracy trade-off preferred.

Training results for COCO 2017 (train/val) after full training schedule with default parameters. Inference results include bounding boxes post-processing for a batch size of 1.

Installation

For best performance, we encourage using the latest PyTorch NGC docker container:

nvidia-docker run --rm --ipc=host -it nvcr.io/nvidia/pytorch:19.05-py3

From the container, simply install retinanet using pip:

pip install --no-cache-dir git+https://github.com/nvidia/retinanet-examples

Or you can clone this repository, build and run your own image:

git clone https://github.com/nvidia/retinanet-examples
docker build -t retinanet:latest retinanet/
nvidia-docker run --rm --ipc=host -it retinanet:latest

Usage

Training, inference, evaluation and model export can be done through the retinanet utility.

For more details refer to the INFERENCE and TRAINING documentation.

Training

Train a detection model on COCO 2017 from pre-trained backbone:

retinanet train retinanet_rn50fpn.pth --backbone ResNet50FPN \
    --images /coco/images/train2017/ --annotations /coco/annotations/instances_train2017.json \
    --val-images /coco/images/val2017/ --val-annotations /coco/annotations/instances_val2017.json

Fine Tuning

Fine-tune a pre-trained model on your dataset. In the example below we use Pascal VOC with JSON annotations:

retinanet train model_mydataset.pth \
    --fine-tune retinanet_rn50fpn.pth \
    --classes 20 --iters 10000 --val-iters 1000 --lr 0.0005 \
    --resize 512 --jitter 480 640 --images /voc/JPEGImages/ \
    --annotations /voc/pascal_train2012.json --val-annotations /voc/pascal_val2012.json

Note: the shorter side of the input images will be resized to resize as long as the longer side doesn't get larger than max-size. During training, the images will be randomly randomly resized to a new size within the jitter range.

Inference

Evaluate your detection model on COCO 2017:

retinanet infer retinanet_rn50fpn.pth --images /coco/images/val2017/ --annotations /coco/annotations/instances_val2017.json

Run inference on your dataset:

retinanet infer retinanet_rn50fpn.pth --images /dataset/val --output detections.json

Optimized Inference with TensorRT

For faster inference, export the detection model to an optimized FP16 TensorRT engine:

retinanet export model.pth engine.plan

Note: for older versions of TensorRT (prior to TensorRT 5.1 / 19.03 containers) the ONNX opset version should be specified (using --opset 8 for instance).

Evaluate the model with TensorRT backend on COCO 2017:

retinanet infer engine.plan --images /coco/images/val2017/ --annotations /coco/annotations/instances_val2017.json

INT8 Inference with TensorRT

For even faster inference, do INT8 calibration to create an optimized INT8 TensorRT engine:

retinanet export model.pth engine.plan --int8 --calibration-images /coco/images/val2017/

This will create an INT8CalibrationTable file that can be used to create INT8 TensorRT engines for the same model later on without needing to do calibration.

Or create an optimized INT8 TensorRT engine using a cached calibration table:

retinanet export model.pth engine.plan --int8 --calibration-table /path/to/INT8CalibrationTable

Datasets

RetinaNet supports annotations in the COCO JSON format. When converting the annotations from your own dataset into JSON, the following entries are required:

{
    "images": [{
        "id" : int,
        "file_name" : str
    }],
    "annotations": [{
        "id" : int,
        "image_id" : int, 
        "category_id" : int,
        "bbox" : [x, y, w, h]
    }],
    "categories": [{
        "id" : int
    ]}
}

References

• Focal Loss for Dense Object Detection. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár. ICCV, 2017.
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, Kaiming He. June 2017.
• Feature Pyramid Networks for Object Detection. Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, Serge Belongie. CVPR, 2017.
• Deep Residual Learning for Image Recognition. Kaiming He, Xiangyu Zhang, Shaoqing Renm Jian Sun. CVPR, 2016.