Special made for the NPU, see Q-engineering deep learning examples
All models are quantized to int8, unless otherwise noted.
demo | model_name | RK3588 | RK3566/68 |
---|---|---|---|
yolov5 | yolov5s_relu | 50.0 | 14.8 |
yolov5n | 58.8 | 19.5 | |
yolov5s | 37.7 | 11.7 | |
yolov5m | 16.2 | 5.7 | |
yolov6 | yolov6n | 63.0 | 18.0 |
yolov6s | 29.5 | 8.1 | |
yolov6m | 15.4 | 4.5 | |
yolov7 | yolov7-tiny | 53.4 | 16.1 |
yolov7 | 9.4 | 3.4 | |
yolov8 | yolov8n | 53.1 | 18.2 |
yolov8s | 28.5 | 8.9 | |
yolov8m | 12.1 | 4.4 | |
yolov10 | yolov10n | 35.1 | 12.5 |
yolov8s | 23.4 | 7.3 | |
yolov8m | 9.7 | 3.4 | |
yolov8x | 5.1 | 1.8 | |
yolox | yolox_s | 30.0 | 10.0 |
yolox_m | 12.9 | 4.8 | |
ppyoloe | ppyoloe_s | 28.8 | 9.2 |
ppyoloe_m | 13.1 | 5.04 | |
yolov5_seg | yolov5n-seg | 9.4 | 1.04 |
yolov5s-seg | 7.8 | 0.87 | |
yolov5m-seg | 6.1 | 0.71 | |
yolov8_seg | yolov8n-seg | 8.9 | 0.91 |
yolov8s-seg | 7.3 | 0.87 | |
yolov8m-seg | 4.5 | 0.7 | |
ppseg | ppseg_lite_1024x512 | 27.5 | 2.4 |
RetinaFace | RetinaFace_mobile3201 | 243.6 | 88.5 |
RetinaFace_resnet50_3201 | 43.4 | 11.8 | |
PPOCR-Det | ppocrv4_det2 | 31.5 | 15.1 |
PPOCR-Rec | ppocrv4_rec3 | 35.7 | 17.3 |
1 Input size 320x320
2 Input size 480x480
3 Input size 48x320, FP16
YoloV10 use layer operations yet unknown to Rock NPU toolset.
To let YoloV10 run on an NPU framework, kaylorchen has modified the onnx file by replacing these layers.
At the same time, he expanded the original output to three.
Until the NPU framework fully supports onnx opset 13, we face these limitations.
To run the application, you have to:
$ sudo apt-get install codeblocks
)Start with the usual
$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get install cmake wget curl
Follow the Raspberry Pi 4 guide.
$ git clone https://github.com/airockchip/rknn-toolkit2.git
We only use a few files.
rknn-toolkit2-master
│
└── rknpu2
│
└── runtime
│
└── Linux
│
└── librknn_api
├── aarch64
│ └── librknnrt.so
└── include
├── rknn_api.h
├── rknn_custom_op.h
└── rknn_matmul_api.h
$ cd ~/rknn-toolkit2-master/rknpu2/runtime/Linux/librknn_api/aarch64
$ sudo cp ./librknnrt.so /usr/local/lib
$ cd ~/rknn-toolkit2-master/rknpu2/runtime/Linux/librknn_api/include
$ sudo cp ./rknn_* /usr/local/include
Save 2 GB of disk space by removing the toolkit. We do not need it anymore.
$ cd ~
$ sudo rm -rf ./rknn-toolkit2-master
To extract and run the network in Code::Blocks
$ mkdir *MyDir* <br/>
$ cd *MyDir* <br/>
$ git clone https://github.com/Qengineering/YoloV10-NPU.git <br/>
You can use Code::Blocks.
Or use Cmake.
$ cd *MyDir*
$ mkdir build
$ cd build
$ cmake ..
$ make -j4
Make sure you use the model fitting your system.
More info or if you want to connect a camera to the app, follow the instructions at Hands-On.
A more than special thanks to kaylorchen, who adapted the onnx model to the NPU framework.