search

wzxzhuxi / rknn-3588-npu-yolo-accelerate

rknn-3588部署yolov5，利用线程池实现npu推理加速；Deploying YOLOv5 on RKNN-3588, utilizing a thread pool to achieve NPU inference acceleration.
Apache License 2.0
43 stars 5 forks source link

大佬 如何实现实时摄像头检测 #8

Open zzx222 opened 1 month ago

zzx222 commented 1 month ago

大佬，您的项目我在香橙派3588上编译出来了。然后我想用自己训练好的模型放上去，实现开启摄像头实时检测。代码中有这部分吗

wzxzhuxi commented 1 month ago

要修改一下代码：在src/yolo_detect.cpp文件中修改函数：void read_stream(const char video_file)，增加判断语句检测命令行传入视频地址是否为0即可，具体实现如下： void read_stream(const char video_file){ cv::VideoCapture cap;

// 判断是否为摄像头设备编号
if (std::string(video_file) == "0")
{
    // 打开摄像头设备
    cap.open(0);  // 0 表示默认摄像头
}
else
{
    // 打开视频文件
    cap.open(video_file);

} 这样，你在命令行启动的时候吧视频地址替换为0，就可以检测摄像头数据了（终于不是编译错误了233333333333）

zzx222 commented 1 month ago

要修改一下代码：在src/yolo_detect.cpp文件中修改函数：void read_stream（const char video_file），增加判断语句检测命令行传入视频地址是否为0即可，具体实现如下： void read_stream（const char video_file）{ cv：：VideoCapture cap;

// 判断是否为摄像头设备编号
if (std::string(video_file) == "0")
{
    // 打开摄像头设备
    cap.open(0);  // 0 表示默认摄像头
}
else
{
    // 打开视频文件
    cap.open(video_file);

}这样，你在命令行启动的时候吧视频地址替换为0，就可以检测摄像头数据了（终于不是编译错误了233333333333）

哥再帮我看一下这个问题，我先试一下模型跑视频，跑项目自带视频是能跑通的。但是用自己模型跑自己视频会有问题 (rknn1.5) bingda@robot-opi5:~/rknn-3588$ ./yolorun.sh [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] Video size: 544 x 960, fps: 30 rgaapi version 1.9.1[4] ./yolorun.sh: line 8: 7583 Segmentation fault ./build/yolov5_thread_pool $MODEL_FILE1 $VIDEO_FILE1 4 All YOLOv5 video processing completed.

这是模型有问题吗，我转rknn时候用的是1.5.0版本

wzxzhuxi commented 1 month ago

要修改一下代码：在src/yolo_detect.cpp文件中修改函数：void read_stream（const char video_file），增加判断语句检测命令行传入视频地址是否为0即可，具体实现如下： void read_stream（const char video_file）{ cv：：VideoCapture cap;

// 判断是否为摄像头设备编号
if (std::string(video_file) == "0")
{
    // 打开摄像头设备
    cap.open(0);  // 0 表示默认摄像头
}
else
{
    // 打开视频文件
    cap.open(video_file);

}这样，你在命令行启动的时候吧视频地址替换为0，就可以检测摄像头数据了（终于不是编译错误了233333333333）

哥再帮我看一下这个问题，我先试一下模型跑视频，跑项目自带视频是能跑通的。但是用自己模型跑自己视频会有问题 (rknn1.5) bingda@robot-opi5:~/rknn-3588$ ./yolorun.sh [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] rknn_init success! [NN_INFO] RKNN API version: 1.5.3b6 (181ec8d8b@2023-09-12T17:11:43) [NN_INFO] RKNN Driver version: 0.8.2 [NN_INFO] model input num: 1, output num: 3 [NN_INFO] input tensors: [NN_INFO] index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] output tensors: [NN_INFO] index=0, name=output0, n_dims=4, dims=[1, 18, 80, 80], n_elems=115200, size=115200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=1, name=275, n_dims=4, dims=[1, 18, 40, 40], n_elems=28800, size=28800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] index=2, name=277, n_dims=4, dims=[1, 18, 20, 20], n_elems=7200, size=7200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922 [NN_INFO] Video size: 544 x 960, fps: 30 rgaapi version 1.9.1[4] ./yolorun.sh: line 8: 7583 Segmentation fault ./build/yolov5_thread_pool $MODEL_FILE1 $VIDEO_FILE1 4 All YOLOv5 video processing completed.

这是模型有问题吗，我转rknn时候用的是1.5.0版本

单看日志这是段错误啊，内存方面的错误，info输出的模型初始化成功，图像边界在预处理的时候也拉成640x640了，你换原作者的项目试试吧：https://github.com/leafqycc/rknn-cpp-Multithreading?tab=readme-ov-file。