fatal error. Please report to developers. Log: 'PlgSrcMipi' '981'

[Neo-cyber-hubb]

Describe the bug

My device is OAK-FFC-4P. I rewrite the source file "depthai_examples/ros1_src/rgb_publisher.cpp" and the launch file "depthai_examples/launch/rgb_publisher.launch" to publish images from four different camera "rgb camd left right". The new files are named "camera_publisher.cpp" and "camera_publisher.launch". I'm running the following command:

roslaunch depthai_examples camera_publisher.launch

In most cases, it is normal. But, sometimes, I get this error:

fatal error. Please report to developers. Log: 'PlgSrcMipi' '981'

This is my source file "depthai_examples/ros1_src/camera_publisher.cpp".

`

include "ros/ros.h"

include

include

// #include "utility.hpp"

include "sensor_msgs/Image.h"

include <camera_info_manager/camera_info_manager.h>

// Inludes common necessary includes for development using depthai library

include "depthai/depthai.hpp"

include <depthai_bridge/BridgePublisher.hpp>

include <depthai_bridge/ImageConverter.hpp>

include <opencv2/opencv.hpp>

dai::Pipeline createPipeline(){ dai::Pipeline pipeline; auto colorCam_left = pipeline.create(); auto colorCam_right = pipeline.create(); auto colorCam_rgb = pipeline.create(); auto colorCam_camd = pipeline.create();

auto xlinkOut_left = pipeline.create<dai::node::XLinkOut>();
auto xlinkOut_right = pipeline.create<dai::node::XLinkOut>();
auto xlinkOut_rgb = pipeline.create<dai::node::XLinkOut>();
auto xlinkOut_camd = pipeline.create<dai::node::XLinkOut>();

xlinkOut_left->setStreamName("video_left");
xlinkOut_right->setStreamName("video_right");
xlinkOut_rgb->setStreamName("video_rgb");
xlinkOut_camd->setStreamName("video_camd");

colorCam_left->setPreviewSize(600, 600);
colorCam_left->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1200_P);
colorCam_left->setInterleaved(false);
colorCam_left->setBoardSocket(dai::CameraBoardSocket::LEFT); //控制这个pipeline接哪个相机，参数：RGB LEFT RIGHT CAM_D
colorCam_left->setFps(10); //设置频率率fps

colorCam_right->setPreviewSize(300, 300);
colorCam_right->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1200_P);
colorCam_right->setInterleaved(false);
colorCam_right->setBoardSocket(dai::CameraBoardSocket::RIGHT); //控制这个pipeline接哪个相机，参数：RGB LEFT RIGHT CAM_D
colorCam_right->setFps(10); //设置频率率fps

colorCam_rgb->setPreviewSize(300, 300);
colorCam_rgb->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1200_P);
colorCam_rgb->setInterleaved(false);
colorCam_rgb->setBoardSocket(dai::CameraBoardSocket::RGB); //控制这个pipeline接哪个相机，参数：RGB LEFT RIGHT CAM_D
colorCam_rgb->setFps(10); //设置频率率fps

colorCam_camd->setPreviewSize(300, 300);
colorCam_camd->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1200_P);
colorCam_camd->setInterleaved(false);
colorCam_camd->setBoardSocket(dai::CameraBoardSocket::CAM_D); //控制这个pipeline接哪个相机，参数：RGB LEFT RIGHT CAM_D
colorCam_camd->setFps(10); //设置频率率fps

// Link plugins CAM -> XLINK
colorCam_left->video.link(xlinkOut_left->input);
colorCam_right->video.link(xlinkOut_right->input);
colorCam_rgb->video.link(xlinkOut_rgb->input);
colorCam_camd->video.link(xlinkOut_camd->input);

return pipeline;

}

int main(int argc, char** argv){

ros::init(argc, argv, "camera_node");
ros::NodeHandle pnh("~");

std::string tfPrefix;
std::string camera_param_uri;
int badParams = 0;

badParams += !pnh.getParam("tf_prefix", tfPrefix);
badParams += !pnh.getParam("camera_param_uri", camera_param_uri);

if (badParams > 0)
{
    throw std::runtime_error("Couldn't find one of the parameters");
}

dai::Pipeline pipeline = createPipeline();
dai::Device device(pipeline);

std::shared_ptr<dai::DataOutputQueue> imgQueue_left = device.getOutputQueue("video_left", 30, false);
std::shared_ptr<dai::DataOutputQueue> imgQueue_right = device.getOutputQueue("video_right", 30, false);
std::shared_ptr<dai::DataOutputQueue> imgQueue_rgb = device.getOutputQueue("video_rgb", 30, false);
std::shared_ptr<dai::DataOutputQueue> imgQueue_camd = device.getOutputQueue("video_camd", 30, false);

std::string left_uri = camera_param_uri + "/" + "left.yaml";
std::string right_uri = camera_param_uri + "/" + "right.yaml";
std::string rgb_uri = camera_param_uri + "/" + "rgb.yaml";
std::string camd_uri = camera_param_uri + "/" + "camd.yaml";

dai::rosBridge::ImageConverter leftConverter(tfPrefix + "_left_camera_optical_frame", false);
dai::rosBridge::ImageConverter rightConverter(tfPrefix + "_right_camera_optical_frame", false);
dai::rosBridge::ImageConverter rgbConverter(tfPrefix + "_rgb_camera_optical_frame", false);
dai::rosBridge::ImageConverter camdConverter(tfPrefix + "_camd_camera_optical_frame", false);

dai::rosBridge::BridgePublisher<sensor_msgs::Image, dai::ImgFrame> leftPublish(imgQueue_left,
                                                                              pnh, 
                                                                              std::string("left/image"),
                                                                              std::bind(&dai::rosBridge::ImageConverter::toRosMsg, 
                                                                              &leftConverter, // since the converter has the same frame name
                                                                                              // and image type is also same we can reuse it
                                                                              std::placeholders::_1, 
                                                                              std::placeholders::_2) , 
                                                                              30,
                                                                              left_uri,
                                                                              "left");

dai::rosBridge::BridgePublisher<sensor_msgs::Image, dai::ImgFrame> rightPublish(imgQueue_right,
                                                                              pnh,
                                                                              std::string("right/image"),
                                                                              std::bind(&dai::rosBridge::ImageConverter::toRosMsg,
                                                                                        &rightConverter, // since the converter has the same frame name
                                                                                      // and image type is also same we can reuse it
                                                                                        std::placeholders::_1,
                                                                                        std::placeholders::_2) ,
                                                                              30,
                                                                              right_uri,
                                                                              "right");

dai::rosBridge::BridgePublisher<sensor_msgs::Image, dai::ImgFrame> rgbPublish(imgQueue_rgb,
                                                                                pnh,
                                                                                std::string("rgb/image"),
                                                                                std::bind(&dai::rosBridge::ImageConverter::toRosMsg,
                                                                                          &rgbConverter, // since the converter has the same frame name
                                                                                        // and image type is also same we can reuse it
                                                                                          std::placeholders::_1,
                                                                                          std::placeholders::_2) ,
                                                                                30,
                                                                                rgb_uri,
                                                                                "rgb");

dai::rosBridge::BridgePublisher<sensor_msgs::Image, dai::ImgFrame> camdPublish(imgQueue_camd,
                                                                                pnh,
                                                                                std::string("camd/image"),
                                                                                std::bind(&dai::rosBridge::ImageConverter::toRosMsg,
                                                                                          &camdConverter, // since the converter has the same frame name
                                                                                        // and image type is also same we can reuse it
                                                                                          std::placeholders::_1,
                                                                                          std::placeholders::_2) ,
                                                                                30,
                                                                                camd_uri,
                                                                                "camd");

leftPublish.addPublisherCallback();
rightPublish.addPublisherCallback();
rgbPublish.addPublisherCallback();
camdPublish.addPublisherCallback();
ros::spin();

return 0;

} `

This is my launch file "depthai_examples/launch/camera_publisher.launch".

`<?xml version="1.0"?>

`

[Neo-cyber-hubb]

Besides, I have another question. For one of the four camera, the ros node would publish several topics like "/camera_publisher/rgb/image", "/camera_publisher/rgb/image/compressed", "/camera_publisher/rgb/depth". How can I change the code so that this ros node only publish the topic "/camera_publisher/rgb/image".

Looking forward to your reply. Thanks!

[Serafadam]

Hi, those additional publishers are created by image_transport ROS mechanism for publishing image, it shouldn't impact anything, you can either edit bridge publisher code or create your own publisher to set up that clean, though then you would have to publish camera info separately (this is done in depthai_ros_driver if you want to see an example). As for camera error, could you provide additional logs by running export DEPTHAI_DEBUG=1 before launching the files? Also, if you only want to publish image streams you can also use depthai_ros_driver here.

[Serafadam]

Closing due to inactivity, please reopen if you have further questions/remarks.