Installing GstRtspServer and introspection typelib
sudo apt update
sudo apt install python3-gi python3-dev python3-gst-1.0 -y
sudo apt-get install libgstrtspserver-1.0-0 gstreamer1.0-rtsp
For gst-rtsp-server (and other GStreamer stuff) to be accessible in Python through gi.require_version(), it needs to be built with gobject-introspection enabled (libgstrtspserver-1.0-0 is already). Yet, we need to install the introspection typelib package:
sudo apt-get install libgirepository1.0-dev
sudo apt-get install gobject-introspection gir1.2-gst-rtsp-server-1.0
source : YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss
[ ] YOLOv7
/YoloLayer_TRT_v7.0/build/libyolo.so
[x] YOLOv8
The yolov8-pose model conversion route is : YOLOv8 PyTorch model -> ONNX -> TensorRT Engine
Notice !!! :warning: This repository don't support TensorRT API building !!!
yolov8s-pose.pt
https://github.com/ultralytics/ultralytics
wget https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8s-pose.pt
Export Orin ONNX model by ultralytics
You can leave this repo and use the original ultralytics
repo for onnx export.
CLI tools(yolo
command from "ultralytics.com")
pip3 install ultralytics
):yolo export model=yolov8s-pose.pt format=onnx device=0 \
imgsz=640 \
dynamic=true \
simplify=true
After executing the above command, you will get an engine named yolov8s-pose.onnx
too.
Move your Onnx Model to egdge device in specific path
sudo chmod u+rwx -R /opt/nvidia/deepstream/deepstream/samples/models # Add Write and execute permissions
sudo mkdir -p tao_pretrained_models/YOLOv8-TensorRT
sudo chmod u+rwx -R tao_pretrained_models/YOLOv8-TensorRT
mv -v
netron yolov8s-pose.onnx
to view the model architectureYOLOv8-Pose
is 56YOLOv7-Pose
is 57 INPUTS
: (batch, channel, height, width)OUTPUTS
: (batch, anchors, max_outpus)-minShapes --optShapes --maxShapes
to set dynamic batch processing.
cd /opt/nvidia/deepstream/deepstream/samples/models/tao_pretrained_models/YOLOv8-TensorRT
sudo /usr/src/tensorrt/bin/trtexec --verbose \
--onnx=yolov8s-pose-dy-sim-640.onnx \
--fp16 \
--workspace=4096 \
--minShapes=images:1x3x640x640 \
--optShapes=images:12x3x640x640 \
--maxShapes=images:16x3x640x640 \
--saveEngine=yolov8s-pose-dy-sim-640.engine
/usr/src/tensorrt/bin/trtexec --loadEngine=yolov8s-pose-dy.engine
or test with multi batch for dynamic shaped onnx model
--shapes=spec
Set input shapes for dynamic shapes inference inputs.
/usr/src/tensorrt/bin/trtexec \
--loadEngine=yolov8s-pose-dy-sim-640.engine \
--shapes=images:12x3x640x640
Performance on Jetson(AGX Xavier / AGX Orin) for TensorRT Engine
model | device | size | batch | fps | ms |
---|---|---|---|---|---|
yolov8s-pose.engine | AGX Xavier | 640 | 1 | 40.6 | 24.7 |
yolov8s-pose.engine | AGX Xavier | 640 | 12 | 12.1 | 86.4 |
yolov8s-pose.engine | AGX Orin | 640 | 1 | 258.8 | 4.2 |
yolov8s-pose.engine | AGX Orin | 640 | 12 | 34.8 | 33.2 |
yolov7w-pose.engine* | AGX Xavier | 960 | 1 | 19.0 | 52.1 |
yolov7w-pose.engine* | AGX Orin | 960 | 1 | 61.1 | 16.8 |
yolov7w-pose.pt | AGX Xavier | 960 | 1 | 14.4 | 59.8 |
yolov7w-pose.pt | AGX Xavier | 960 | 1 | 11.8 | 69.4 |
trtexec
command.git clone https://github.com/YunghuiHsu/deepstream-yolo-pose.git
NVInfer with rtsp inputs
python3 deepstream_YOLOv8-Pose_rtsp.py \
-i rtsp://sample_1.mp4 \
rtsp://sample_2.mp4 \
rtsp://sample_N.mp4 \
python3 deepstream_YOLOv8-Pose_rtsp.py \
-i file:///home/ubuntu/video1.mp4 file:///home/ubuntu/video2.mp4 \
-config dstest1_pgie_YOLOv8-Pose_config.txt \
--file-loop
Default RTSP streaming location:
rtsp://<server IP>:8554/ds-test
Note:
1) if -g/--pgie
: uses nvinfer as default. (['nvinfer', 'nvinferserver']).
2) -config/--config-file
: need to be provided for custom models.
3) --file-loop
: option can be used to loop input files after EOS.
4) --conf-thres
: Objec Confidence Threshold
5) --iou-thres
: IOU Threshold for NMS
This sample app is derived from NVIDIA-AI-IOT/deepstream_python_apps/apps and adds customization features
Includes following :
NvDsInferTensorMeta
) [x] Extract the stream metadata, image data from the batched buffer of Gst-nvinfer