Deepstream-YOLO-Pose

System Requirements

• Python 3.8
  • Should be already installed with Ubuntu 20.04
• Ubuntu 20.04
• CUDA 11.4 (Jetson)
• TensorRT 8+

DeepStream 6.x on x86 platform

• Ubuntu 20.04
• CUDA 11.8
• TensorRT 8.5 GA Update 1 (8.5.2.2)
• NVIDIA Driver 525.85.12 (Data center / Tesla series) / 525.105.17 (TITAN, GeForce RTX / GTX series and RTX / Quadro series)
• NVIDIA DeepStream SDK 6.x
• GStreamer 1.16.3
• DeepStream-Yolo

DeepStream 6.x on Jetson platform

• JetPack 5.1.1 / 5.1
• NVIDIA DeepStream SDK
  • Download and install from https://developer.nvidia.com/deepstream-download
• DeepStream-Yolo

Deepstream Python Biding

• Deepstream Python Biding

Gst-python and GstRtspServer

• Installing GstRtspServer and introspection typelib

  sudo apt update
  sudo apt install python3-gi python3-dev python3-gst-1.0 -y
  sudo apt-get install libgstrtspserver-1.0-0 gstreamer1.0-rtsp

  For gst-rtsp-server (and other GStreamer stuff) to be accessible in Python through gi.require_version(), it needs to be built with gobject-introspection enabled (libgstrtspserver-1.0-0 is already). Yet, we need to install the introspection typelib package:

  sudo apt-get install libgirepository1.0-dev
  sudo apt-get install gobject-introspection gir1.2-gst-rtsp-server-1.0

  ---

  Prepare YOLO-Pose Model

  

  source : YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss

• [ ] YOLOv7

  • Gwencong/yolov7-pose-tensorrt
  • nanmi/yolov7-pose
  • support single batch only
  • Some problems with /YoloLayer_TRT_v7.0/build/libyolo.so
    • The detection box is not synchronized with the screen on Jetson

• [x] YOLOv8

Prepare YOLOv8 TensorRT Engine

• Choose yolov8-pose for better operator optimization of ONNX model
• Base on triple-Mu/YOLOv8-TensorRT/Pose.md

• The yolov8-pose model conversion route is : YOLOv8 PyTorch model -> ONNX -> TensorRT Engine

  Notice !!! :warning: This repository don't support TensorRT API building !!!

0. Get yolov8s-pose.pt

https://github.com/ultralytics/ultralytics

wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s-pose.pt

1. Pytorch Model to Onnx Model

• Export Orin ONNX model by ultralytics You can leave this repo and use the original ultralytics repo for onnx export.

• CLI tools(yolo command from "ultralytics.com")

  • Recommended in your server to get faster speed :zap:
  • ref : ultralytics.com/modes/export
  • Usage(after pip3 install ultralytics):

  yolo export model=yolov8s-pose.pt format=onnx device=0 \
            imgsz=640 \
            dynamic=true \
            simplify=true

  After executing the above command, you will get an engine named yolov8s-pose.onnx too.

• Move your Onnx Model to egdge device in specific path

  • put model on your edge device

    
    sudo chmod u+rwx -R /opt/nvidia/deepstream/deepstream/samples/models # Add Write and execute permissions 
    sudo mkdir -p tao_pretrained_models/YOLOv8-TensorRT 
    sudo chmod u+rwx -R tao_pretrained_models/YOLOv8-TensorRT 

  mv -v /opt/nvidia/deepstream/deepstream/samples/models/tao_pretrained_models/YOLOv8-TensorRT/yolov8s-pose-dy-sim-640.onnx

[Optional] Execute netron yolov8s-pose.onnx to view the model architecture

• Check Model Ouputs
  • Note that the number of anchors for YOLOv8-Pose is 56
  • bbox(4) + confidence(1) + keypoints(3 x 17) = 4 + 1 + 0 + 51 = 56
  • The number of anchors of YOLOv7-Pose is 57
  • bbox(4) + confidence(1) + cls(1) + keypoints(3 x 17) = 4 + 1 + 1 + 51 = 57
• Model registration information of YOLOv8S-Pose
  • INPUTS : (batch, channel, height, width)
  • OUTPUTS : (batch, anchors, max_outpus)

2. Onnx to TensorRT Engine with dynamic_batch

• :warning: Must be bound to a hardware device, please put it on your edge device(It's a long wait :hourglass:)
• Specify parameters such as -minShapes --optShapes --maxShapes to set dynamic batch processing.

  cd /opt/nvidia/deepstream/deepstream/samples/models/tao_pretrained_models/YOLOv8-TensorRT 
  sudo /usr/src/tensorrt/bin/trtexec --verbose \
    --onnx=yolov8s-pose-dy-sim-640.onnx \
    --fp16 \
    --workspace=4096 \
    --minShapes=images:1x3x640x640 \
    --optShapes=images:12x3x640x640 \
    --maxShapes=images:16x3x640x640 \
    --saveEngine=yolov8s-pose-dy-sim-640.engine

3. Test and Check Tensortrt Engine

/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8s-pose-dy.engine

• or test with multi batch for dynamic shaped onnx model

  • --shapes=spec Set input shapes for dynamic shapes inference inputs.

    /usr/src/tensorrt/bin/trtexec  \
    --loadEngine=yolov8s-pose-dy-sim-640.engine \
    --shapes=images:12x3x640x640 

  • Performance on Jetson(AGX Xavier / AGX Orin) for TensorRT Engine

• * yolov7w-pose with yolo layer tensorrt plugin from (nanmi/yolov7-pose).NMS not included。Single batch and image_size 960 only.
• test .engine(TensorRT) model with trtexec command.
  
• test .pt model with Pytorch (with 15s video) for baseline.
• NMS not included in all test

Basic usage

Download Ripository

git clone https://github.com/YunghuiHsu/deepstream-yolo-pose.git

To run the app with default settings:

• NVInfer with rtsp inputs

   python3 deepstream_YOLOv8-Pose_rtsp.py \ 
      -i  rtsp://sample_1.mp4 \
          rtsp://sample_2.mp4 \ 
          rtsp://sample_N.mp4  \

  • eg: loop with local file inputs

    python3 deepstream_YOLOv8-Pose_rtsp.py \
      -i file:///home/ubuntu/video1.mp4 file:///home/ubuntu/video2.mp4 \
      -config dstest1_pgie_YOLOv8-Pose_config.txt \
      --file-loop

• Default RTSP streaming location:

  • rtsp://<server IP>:8554/ds-test
  • VLC Player on client suggested(Camera Streaming and Multimedia)

  Note: 1) if -g/--pgie : uses nvinfer as default. (['nvinfer', 'nvinferserver']). 2) -config/--config-file : need to be provided for custom models. 3) --file-loop : option can be used to loop input files after EOS. 4) --conf-thres : Objec Confidence Threshold 5) --iou-thres : IOU Threshold for NMS

  This sample app is derived from NVIDIA-AI-IOT/deepstream_python_apps/apps and adds customization features

• Includes following :

  • [x] Accepts multiple sources
  • [x] Dynamic batch model(YOLO-POSE)
  • [x] Accepts RTSP stream as input and gives out inference as RTSP stream
  • [x] NVInfer GPU inference engine
  • [ ] NVInferserver GPU inference engine(Not yet tested)
  • [x] MultiObjectTracker(NVTracker)
  • [x] Automatically adjusts the tensor shape of the loaded input and output (NvDsInferTensorMeta)

  • [x] Extract the stream metadata, image data from the batched buffer of Gst-nvinfer

    

    source : deepstream-imagedata-multistream

    Acknowledgements

  • YOLOv5
  • YOLOv7
  • YOLOv8
  • TexasInstruments/edgeai-yolov5
  • triple-Mu/YOLOv8-TensorRT
  • marcoslucianops/DeepStream-Yolo
  • Gwencong/yolov7-pose-tensorrt
  • nanmi/yolov7-pose

Reference

NVIDIA DeepStream SDK API Reference/NvDsInferTensorMeta Struct Reference

DEEPSTREAM PYTHON API REFERENCE/NvDsInfer

Using a Custom Model with DeepStream