search

PaddlePaddle / PaddleDetection

Object Detection toolkit based on PaddlePaddle. It supports object detection, instance segmentation, multiple object tracking and real-time multi-person keypoint detection.
Apache License 2.0
12.41k stars 2.85k forks source link

poor performance on Rockchip RK3588 (incl int8, Paddle Lite) #8730

Open debugmenot opened 7 months ago

debugmenot commented 7 months ago

问题确认 Search before asking

请提出你的问题 Please ask your question

Hi guys! PaddlePaddle is awesome! we are using it for a long time already. now, we are trying to migrate some of our models to edge devices. We've tried our own trained models (various sizes of picodet) and pretrained picodet models from this repo with PP Lite, including int8 models on RockChip RK3588 board (4 threads). Just for example, quantized int8 320x320 nanodet-m give us around 20ms inference time with ONNXRuntime... Our hope was that picodet will be faster, but... picodet gives us:

picodet_s_416_coco_lcnet_quant: Preprocess time: 7.602000 ms Prediction time: 39.634799 ms Postprocess time: 0.838000 ms

picodet_xs_320_coco_lcnet_fp16: Preprocess time: 7.612000 ms Prediction time: 29.944400 ms Postprocess time: 0.942000 ms

May be we are doing something wrong? Or this results/performance on RK3588 (4 threads) are expected? We are not using NPU.

Results are obtained with /Paddle-Lite-Demo/object_detection/linux/picodet_detection

Please help!)

LokeZhou commented 4 months ago

Sorry, we have not conducted any speed evaluation on thisRK3588. You can make requests here ：https://github.com/PaddlePaddle/Paddle-Lite