luxonis / depthai-python

DepthAI Python Library
MIT License
361 stars 192 forks source link

Image Manip Node max CPU Fatal error on MSS CPU: trap: 00, address: 00000000' '0' #711

Closed PeterQFR closed 1 year ago

PeterQFR commented 1 year ago

Possibly related to #625

I am running a spatialImageDetector node and as my cameras are mounted 90 degrees to normal I rotate the depth and rgb images 90 degrees using a imagemanip node. when doing so the LeonRT maxes out to 100% and a CPU trap is triggered.

In my debug output from the node:

[INFO] [1669770818.307067979] [depthAI_node]: Initialising DepthAI
[INFO] [1669770818.315620593] [depthAI_node]: Comms Setup
[INFO] [1669770818.315639026] [depthAI_node]: Callbacks Setup
[INFO] [1669770818.315662691] [depthAI_node]: Polling Queues Setup
[1944301051BBED1200] [3.1] [2.926] [system] [info] Memory Usage - DDR: 133.67 / 340.61 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 54.55 / 77.48 MiB, LeonRT Heap: 5.90 / 41.35 MiB
[1944301051BBED1200] [3.1] [2.926] [system] [info] Temperatures - Average: 44.01 °C, CSS: 44.87 °C, MSS 43.96 °C, UPA: 44.64 °C, DSS: 42.58 °C
[1944301051BBED1200] [3.1] [2.926] [system] [info] Cpu Usage - LeonOS 57.11%, LeonRT: 54.64%
[1944301051BBED1200] [3.1] [3.927] [system] [info] Memory Usage - DDR: 133.67 / 340.61 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 54.55 / 77.48 MiB, LeonRT Heap: 5.90 / 41.35 MiB
[1944301051BBED1200] [3.1] [3.927] [system] [info] Temperatures - Average: 44.47 °C, CSS: 44.87 °C, MSS 44.18 °C, UPA: 44.87 °C, DSS: 43.96 °C
[1944301051BBED1200] [3.1] [3.927] [system] [info] Cpu Usage - LeonOS 6.13%, LeonRT: 100.00%
[1944301051BBED1200] [3.1] [4.928] [system] [info] Memory Usage - DDR: 133.67 / 340.61 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 54.55 / 77.48 MiB, LeonRT Heap: 5.90 / 41.35 MiB
[1944301051BBED1200] [3.1] [4.928] [system] [info] Temperatures - Average: 44.41 °C, CSS: 45.77 °C, MSS 44.64 °C, UPA: 44.87 °C, DSS: 42.35 °C
[1944301051BBED1200] [3.1] [4.928] [system] [info] Cpu Usage - LeonOS 5.91%, LeonRT: 100.00%
[1944301051BBED1200] [3.1] [5.929] [system] [info] Memory Usage - DDR: 133.67 / 340.61 MiB, CMX: 2.49 / 2.50 MiB, LeonOS Heap: 54.55 / 77.48 MiB, LeonRT Heap: 5.90 / 41.35 MiB
[1944301051BBED1200] [3.1] [5.929] [system] [info] Temperatures - Average: 44.47 °C, CSS: 45.32 °C, MSS 44.64 °C, UPA: 44.64 °C, DSS: 43.27 °C
[1944301051BBED1200] [3.1] [5.929] [system] [info] Cpu Usage - LeonOS 5.76%, LeonRT: 100.00%
[1944301051BBED1200] [3.1] [7.694] [system] [critical] Fatal error. Please report to developers. Log: 'Fatal error on MSS CPU: trap: 00, address: 00000000' '0'

If I disable using the ImageManipNodes and just pipe depth and rgb preview directly to the spatial detector I do not get an issue with the CPU.

The code works on earlier commit 02c304803aabaced594e8173152c6dedb260cd06

but this error appears on a newer commit on the main branch 690b6a63

I am use a DepthAI Pro.

PeterQFR commented 1 year ago

Example Code that doesn't work: NB depthtemp is a depth node initialised elsewhere.

       xoutRGB_=  pipeline.create<dai::node::XLinkOut>();
        xoutRGB_->setStreamName("rgb_preview");
        xoutRGB_->input.setBlocking(false);
        xoutRGB_->input.setQueueSize(1);
        rgb_ = pipeline.create<dai::node::ColorCamera>();
        rgb_->setFps(1.0f);
        rgb_->setResolution(dai::ColorCameraProperties::SensorResolution::THE_1080_P);
        rgb_->setInterleaved(false);
        rgb_->setColorOrder(dai::ColorCameraProperties::ColorOrder::BGR);

        rgb_->setPreviewSize(416, 416);

        xoutNN_ = pipeline.create<dai::node::XLinkOut>();
        xoutNN_->setStreamName("detections");

        xoutBBDepthMapping_ = pipeline.create<dai::node::XLinkOut>();
        xoutBBDepthMapping_->setStreamName("boundingboxes");

        spatialDetector_ = pipeline.create<dai::node::YoloSpatialDetectionNetwork>();

        spatialDetector_->setConfidenceThreshold(CONFIDENCE_THRESHOLD);
        spatialDetector_->input.setBlocking(false);
        spatialDetector_->inputDepth.setBlocking(false);

        spatialDetector_->setBoundingBoxScaleFactor(0.5);
        spatialDetector_->setDepthLowerThreshold(100);
        spatialDetector_->setDepthUpperThreshold(20000);
        spatialDetector_->setNumClasses(80);
        spatialDetector_->setCoordinateSize(4);
        spatialDetector_->setAnchors({10, 14, 23, 27,37, 58, 81, 82, 135, 169, 344, 319});
        spatialDetector_->setAnchorMasks({{"side26", {1, 2, 3}}, {"side13", {3, 4, 5}}});
        spatialDetector_->setIouThreshold(0.5f);
        spatialDetector_->setBlobPath(path_);

      //Comments here turn on or off rotation of depth image to see if that affects CPU.    
       // depthRotation_ = pipeline.create<dai::node::ImageManip>();
        colorRotation_ = pipeline.create<dai::node::ImageManip>();

       /* depthRotation_->initialConfig.setRotationRadians(M_PI_2);
        depthRotation_->initialConfig.setFrameType(dai::ImgFrame::Type::RAW16);
        depthRotation_->inputImage.setBlocking(false);
        depthRotation_->inputImage.setQueueSize(1);
*/
        colorRotation_->initialConfig.setRotationRadians(M_PI_2);
        colorRotation_->initialConfig.setFrameType(dai::ImgFrame::Type::BGR888p);
        colorRotation_->inputImage.setBlocking(false);
        colorRotation_->inputImage.setQueueSize(1);

        //depth_temp_->depth.link(depthRotation_->inputImage);
        rgb_->preview.link(colorRotation_->inputImage);

        colorRotation_->out.link(spatialDetector_->input);
        //depthRotation_->out.link(spatialDetector_->inputDepth);
        depth_temp_->depth.link(spatialDetector_->inputDepth);

        spatialDetector_->input.setBlocking(false);
        spatialDetector_->input.setQueueSize(1);
        spatialDetector_->inputDepth.setBlocking(false);
        spatialDetector_->inputDepth.setQueueSize(1); 
        spatialDetector_->boundingBoxMapping.link(xoutBBDepthMapping_->input);
        spatialDetector_->out.link(xoutNN_->input);
        spatialDetector_->passthrough.link(xoutRGB_->input);
PeterQFR commented 1 year ago

Actually this is for depthai-core will move the issue there.