search

kaylorchen / rk3588-yolo-demo

The project is a multi-threaded inference demo of Yolo running on the RK3588 platform, which has been adapted for reading video files and camera feeds. The demo uses the Yolov8n model for file inference, with a maximum inference frame rate of up to 100 frames per second.
MIT License
204 stars 29 forks source link

Frames RK3588 #28

Open utya opened 1 week ago

utya commented 1 week ago 
./test_yolo_cpp_bytrack.sh 
environment variable SPDLOG_STDOUT is not set, and set true
environment variable SPDLOG_SYSTEMD is not set, and set false
environment variable SPDLOG_LEVEL is not set, and set default level
[INFO] load lable /root/rk3588-yolo-demo/src/yolov8/model/coco_80_labels_list.txt
[INFO] There are 80 lines
[INFO] rknn_init() is called
[INFO] sdk version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11) driver version: 0.9.3
[INFO] model input num: 1, and output num: 9
[INFO] input tensors:
[INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003921569
[INFO] output tensors:

[INFO] model is NHWC input fmt
[INFO] model input height=640, width=640, channel=3
[INFO] rknn_dup_context() is called
[INFO] sdk version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11) driver version: 0.9.3
[INFO] model input num: 1, and output num: 9

I am running your code, built everything according to the instructions, but I can’t get an FPS higher than 23. Could the operating system be the issue?

kaylorchen commented 1 week ago 
./test_yolo_cpp_bytrack.sh 
environment variable SPDLOG_STDOUT is not set, and set true
environment variable SPDLOG_SYSTEMD is not set, and set false
environment variable SPDLOG_LEVEL is not set, and set default level
[INFO] load lable /root/rk3588-yolo-demo/src/yolov8/model/coco_80_labels_list.txt
[INFO] There are 80 lines
[INFO] rknn_init() is called
[INFO] sdk version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11) driver version: 0.9.3
[INFO] model input num: 1, and output num: 9
[INFO] input tensors:
[INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003921569
[INFO] output tensors:

[INFO] model is NHWC input fmt
[INFO] model input height=640, width=640, channel=3
[INFO] rknn_dup_context() is called
[INFO] sdk version: 1.6.0 (9a7b5d24c@2023-12-13T17:31:11) driver version: 0.9.3
[INFO] model input num: 1, and output num: 9

I am running your code, built everything according to the instructions, but I can’t get an FPS higher than 23. Could the operating system be the issue?

set export SPDLOG_LEVEL=debug, and upload your all the log.

utya commented 1 week ago

@kaylorchen https://gist.github.com/utya/9c6f65c9feea4d22c01a5a424300b8dc maybe kernel 6 is reason?

kaylorchen commented 1 week ago

@kaylorchen https://gist.github.com/utya/9c6f65c9feea4d22c01a5a424300b8dc maybe kernel 6 is reason?

What is your model version? the inference time takes 130ms. 130ms/ 4 threads = 33ms, max fps: 30.

utya commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

kaylorchen commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

utya commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

kaylorchen commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

yes, 1.6.0

utya commented 1 week ago

with one thread model M (2.3.0) [INFO] model type is 2 [INFO] chair @ (347 215 515 438) 0.8828281 [INFO] cup @ (542 221 557 239) 0.59976524 [INFO] laptop @ (220 206 283 251) 0.59976524 [INFO] image count = 878, image res count = 877, delta = 1, duration = 71ms [INFO] rknn_outputs_post_process cost time: 0.232ms [DEBUG] Inference time is 68ms and total time is 70ms

utya commented 1 week ago

with one thread model yolov8n.rknn (1.6.0) [DEBUG] Inference time is 17ms and total time is 19ms [INFO] ImagePostProcess is called [INFO] model type is 2 [INFO] person @ (468 159 555 345) 0.7167225 [INFO] image count = 668, image res count = 664, delta = 4, duration = 43ms [INFO] rknn_outputs_post_process cost time: 0.152ms [DEBUG] Inference time is 17ms and total time is 19ms

Could you advise what else to check? Could it be due to the lack of a cooler? Or because the system is running on a USB flash drive?

kaylorchen commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

with one thread model yolov8n.rknn (1.6.0) [DEBUG] Inference time is 17ms and total time is 19ms [INFO] ImagePostProcess is called [INFO] model type is 2 [INFO] person @ (468 159 555 345) 0.7167225 [INFO] image count = 668, image res count = 664, delta = 4, duration = 43ms [INFO] rknn_outputs_post_process cost time: 0.152ms [DEBUG] Inference time is 17ms and total time is 19ms

Could you advise what else to check? Could it be due to the lack of a cooler? Or because the system is running on a USB flash drive?

This result seems to be correct. I think it is a heat dissipation problem.

kaylorchen commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

with one thread model yolov8n.rknn (1.6.0) [DEBUG] Inference time is 17ms and total time is 19ms [INFO] ImagePostProcess is called [INFO] model type is 2 [INFO] person @ (468 159 555 345) 0.7167225 [INFO] image count = 668, image res count = 664, delta = 4, duration = 43ms [INFO] rknn_outputs_post_process cost time: 0.152ms [DEBUG] Inference time is 17ms and total time is 19ms Could you advise what else to check? Could it be due to the lack of a cooler? Or because the system is running on a USB flash drive?

This result seems to be correct. I think it is a heat dissipation problem.

maybe ... USB

utya commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

with one thread model yolov8n.rknn (1.6.0) [DEBUG] Inference time is 17ms and total time is 19ms [INFO] ImagePostProcess is called [INFO] model type is 2 [INFO] person @ (468 159 555 345) 0.7167225 [INFO] image count = 668, image res count = 664, delta = 4, duration = 43ms [INFO] rknn_outputs_post_process cost time: 0.152ms [DEBUG] Inference time is 17ms and total time is 19ms Could you advise what else to check? Could it be due to the lack of a cooler? Or because the system is running on a USB flash drive?

This result seems to be correct. I think it is a heat dissipation problem.

maybe ... USB

usb camera? logitech c270

utya commented 1 week ago

thanks for your code Initially, I used Ubuntu with kernel 6. But the latest logs are already on kernel 5.

kaylorchen commented 1 week ago

i use your model https://github.com/kaylorchen/rk3588-yolo-demo/blob/master/src/yolov8/model/yolov8n.rknn My camera outputs 30 fps. When I provide a 1920x1080 30 fps video, I get 8 fps. Could you tell me how you achieve 120 fps? What are you testing on?

I think you should use single threaded mode and test your inference speed. may be some hardware issues or your cpu/npu frequency is too low. 130ms inference time for yolov8n model is too long.

Did you use version 1.6.0?

with one thread model yolov8n.rknn (1.6.0) [DEBUG] Inference time is 17ms and total time is 19ms [INFO] ImagePostProcess is called [INFO] model type is 2 [INFO] person @ (468 159 555 345) 0.7167225 [INFO] image count = 668, image res count = 664, delta = 4, duration = 43ms [INFO] rknn_outputs_post_process cost time: 0.152ms [DEBUG] Inference time is 17ms and total time is 19ms Could you advise what else to check? Could it be due to the lack of a cooler? Or because the system is running on a USB flash drive?

This result seems to be correct. I think it is a heat dissipation problem.

maybe ... USB

usb camera? logitech c270

The c270 doesn't have 30fps in low light. I also have a c270 so I know the camera doesn't have enough fps.

utya commented 1 week ago

What affordable options with good FPS would you recommend?