search

luxonis / depthai-core

DepthAI C++ Library
MIT License
235 stars 127 forks source link

`UseHomographyRectification` API doesn't correct lens distortion on-device with 120 FoV lense. #638

Open developer-mayuan opened 2 years ago

developer-mayuan commented 2 years ago

I have a question about useHomographyRectification API of StereoDepth node. According to the documentation (as shown in the figure below), the mesh data should be generated same as opencv's initUndistortRectifyMap function. image

In my setup, I'm currently on lenses with 120 degree FoV, so a sparse mesh should be generated on the edge by it which corrects the lense distortion. But I actually got different rectification results between using this API and providing my own mesh data generated from a slighted modified version of depthai's calibration script. Could this caused because I use a different image resolution at runtime (400P) instead of the resolution used during calibration (800P)?

Rectified images using HomographyRectification(false) (wrong result): Screenshot from 2022-11-10 14-14-39

Rectified images using customized mesh data generated from a modified version of calibration script (correct result): Screenshot from 2022-11-10 14-16-47

The modification we made to the calibration script is in the save_mesh_data() function as following so that we can calibrate camera at 800P and rectify images at 400P at runtime:

def create_save_mesh(self, decimation=2):  # , output_path):

        curr_path = Path(__file__).parent.resolve()
        print("Mesh path")
        print(curr_path)

        map_x_l, map_y_l = cv2.initUndistortRectifyMap(
            self.M1, self.d1, self.R1, self.M2, self.img_shape, cv2.CV_32FC1
        )
        map_x_r, map_y_r = cv2.initUndistortRectifyMap(
            self.M2, self.d2, self.R2, self.M2, self.img_shape, cv2.CV_32FC1
        )

        print("shape of maps")
        print(map_x_l.shape)
        print(map_y_l.shape)
        print(map_x_r.shape)
        print(map_y_r.shape)

        meshCellSize = 40
        mesh_left = []
        mesh_right = []

        for y in range(0, map_x_r.shape[0] + 1, meshCellSize):
            for x in range(0, map_x_r.shape[1] + 1, meshCellSize):
                x_ind = x
                y_ind = y
                if y == map_x_r.shape[0]:
                    y_ind = y - 1
                if x == map_x_r.shape[1]:
                    x_ind = x - 1
                # mesh.append((map_y[y_ind, x_ind], map_x[y_ind, x_ind]))
                mesh_left.append(
                    (
                        map_y_l[y_ind, x_ind] / decimation,
                        map_x_l[y_ind, x_ind] / decimation,
                    )
                )
                mesh_right.append(
                    (
                        map_y_r[y_ind, x_ind] / decimation,
                        map_x_r[y_ind, x_ind] / decimation,
                    )
                )

        mesh_left = np.array(mesh_left).astype(np.float32)
        mesh_right = np.array(mesh_right).astype(np.float32)
        left_mesh_fpath = str(curr_path) + "/../resources/left_mesh.calib"
        right_mesh_fpath = str(curr_path) + "/../resources/right_mesh.calib"
        mesh_left.tofile(left_mesh_fpath)
        mesh_right.tofile(right_mesh_fpath)
alex-luxonis commented 2 years ago

Can you attach the device calibration data, we can have a look. This example can be used to retrieve it: https://github.com/luxonis/depthai-core/blob/main/examples/calibration/calibration_dump.cpp

developer-mayuan commented 2 years ago

Hi @alex-luxonis Here is the calibration data I believe (we rotate the camera stream before calibration, so the image resolution is 800*1280):

{
    "batchName": "",
    "batchTime": 0,
    "boardConf": "",
    "boardCustom": "",
    "boardName": "MY_BOARD",
    "boardOptions": 0,
    "boardRev": "R2M2E2",
    "cameraData": [
        [
            2,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -5.180891036987305,
                    8.956652641296387,
                    0.0003124542417936027,
                    1.3796477105643135e-05,
                    1.4717915058135986,
                    -4.844494819641113,
                    7.172787666320801,
                    4.7251362800598145,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ],
                        [
                            0.0,
                            0.0,
                            0.0
                        ]
                    ],
                    "specTranslation": {
                        "x": 0.0,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": -1,
                    "translation": {
                        "x": 0.0,
                        "y": 0.0,
                        "z": 0.0
                    }
                },
                "height": 1280,
                "intrinsicMatrix": [
                    [
                        601.024658203125,
                        0.0,
                        407.5490417480469
                    ],
                    [
                        0.0,
                        600.6100463867188,
                        643.2725219726563
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 120.0,
                "width": 800
            }
        ],
        [
            1,
            {
                "cameraType": 0,
                "distortionCoeff": [
                    -29.545913696289063,
                    228.75546264648438,
                    0.00014689743693452328,
                    -0.0006457889685407281,
                    50.74214553833008,
                    -29.196752548217773,
                    218.35711669921875,
                    132.84185791015625,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0,
                    0.0
                ],
                "extrinsics": {
                    "rotationMatrix": [
                        [
                            0.998506486415863,
                            -0.003170691430568695,
                            0.054541438817977905
                        ],
                        [
                            0.003490656381472945,
                            0.9999772310256958,
                            -0.005772195290774107
                        ],
                        [
                            -0.054521895945072174,
                            0.005953959655016661,
                            0.9984948039054871
                        ]
                    ],
                    "specTranslation": {
                        "x": -7.5,
                        "y": 0.0,
                        "z": 0.0
                    },
                    "toCameraSocket": 2,
                    "translation": {
                        "x": -8.26642894744873,
                        "y": -0.08183234184980392,
                        "z": 0.41030120849609375
                    }
                },
                "height": 1280,
                "intrinsicMatrix": [
                    [
                        601.441650390625,
                        0.0,
                        397.5523681640625
                    ],
                    [
                        0.0,
                        601.0449829101563,
                        628.9996337890625
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "lensPosition": 0,
                "specHfovDeg": 120.0,
                "width": 800
            }
        ]
    ],
    "hardwareConf": "",
    "imuExtrinsics": {
        "rotationMatrix": [
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ],
            [
                0.0,
                0.0,
                0.0
            ]
        ],
        "specTranslation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        },
        "toCameraSocket": -1,
        "translation": {
            "x": 0.0,
            "y": 0.0,
            "z": 0.0
        }
    },
    "miscellaneousData": [],
    "productName": "",
    "stereoRectificationData": {
        "leftCameraSocket": 1,
        "rectifiedRotationLeft": [
            [
                0.9999672770500183,
                0.00642469571903348,
                0.0049180760979652405
            ],
            [
                -0.006409789901226759,
                0.9999748468399048,
                -0.00304059823974967
            ],
            [
                -0.004937487188726664,
                0.003008974948897958,
                0.9999833106994629
            ]
        ],
        "rectifiedRotationRight": [
            [
                0.9987216591835022,
                0.009886703453958035,
                -0.04957118630409241
            ],
            [
                -0.009736666455864906,
                0.9999472498893738,
                0.0032672618981450796
            ],
            [
                0.04960087314248085,
                -0.002780427224934101,
                0.998765230178833
            ]
        ],
        "rightCameraSocket": 2
    },
    "version": 7
}
SzabolcsGergely commented 2 years ago

@developer-mayuan if you rotate the cameras during calibration, then you have to rotate them prior to sending them to the stereo node, and set input resolution explicitly.

developer-mayuan commented 2 years ago

Hi @szabi-luxonis: Thanks for your suggestion, and yes, this is exactly what we have done for our runtime pipeline setup. We matched it with the pipeline we setup for calibration, otherwise the result we got will be completely wrong, instead of something I posted here.

EDIT 1: The image below shows how my pipeline looks like: Screenshot from 2022-11-11 10-31-09

EDIT 2: This is the debug logging when use UseHomographyRectification for undistortion:

[2022-11-11 18:35:01.694] [debug] Device - OpenVINO version: 2022.1
[2022-11-11 18:35:01.700] [debug] Device - BoardConfig: {"emmc":null,"gpio":[],"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"maxSpeed":4,"pid":63035,"vid":999},"usb3PhyInternalClock":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null} 
libnop:
0000: b9 0e b9 05 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 b9 02 00 01 ba 00 be be bb 00 bb 00 be be be
0020: be be be be
[2022-11-11 18:35:01.835] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.21.tar.xz' open: 14ms, archive read: 140ms
[2022-11-11 18:35:02.224] [debug] Resources - Archive 'depthai-device-fwp-b3aeaf23ff5857fc8f79d412ceefc08da23e7aad.tar.xz' open: 3ms, archive read: 544ms
[2022-11-11 18:35:03.250] [debug] Searching for booted device: DeviceInfo(name=1.10.1, mxid=1844301081DE841200, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Memory Usage - DDR: 0.12 / 340.61 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 7.23 / 77.48 MiB, LeonRT Heap: 2.89 / 41.35 MiB
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Temperatures - Average: 38.12 °C, CSS: 40.28 °C, MSS 37.71 °C, UPA: 37.48 °C, DSS: 37.01 °C
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Cpu Usage - LeonOS 62.90%, LeonRT: 1.39%
[2022-11-11 18:35:04.517] [debug] Schema dump: {"connections":[{"node1Id":5,"node1Output":"out","node1OutputGroup":"","node2Id":6,"node2Input":"in","node2InputGroup":""},{"node1Id":3,"node1Output":"out","node1OutputGroup":"","node2Id":5,"node2Input":"inputImage","node2InputGroup":""},{"node1Id":4,"node1Output":"rectifiedRight","node1OutputGroup":"","node2Id":8,"node2Input":"in","node2InputGroup":""},{"node1Id":4,"node1Output":"rectifiedLeft","node1OutputGroup":"","node2Id":7,"node2Input":"in","node2InputGroup":""},{"node1Id":4,"node1Output":"depth","node1OutputGroup":"","node2Id":9,"node2Input":"in","node2InputGroup":""},{"node1Id":3,"node1Output":"out","node1OutputGroup":"","node2Id":4,"node2Input":"right","node2InputGroup":""},{"node1Id":2,"node1Output":"out","node1OutputGroup":"","node2Id":4,"node2Input":"left","node2InputGroup":""},{"node1Id":1,"node1Output":"isp","node1OutputGroup":"","node2Id":3,"node2Input":"inputImage","node2InputGroup":""},{"node1Id":0,"node1Output":"isp","node1OutputGroup":"","node2Id":2,"node2Input":"inputImage","node2InputGroup":""}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","video"],{"blocking":false,"group":"","name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,23,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,1,255,0,1,0,129,44,1,129,44,1,255,255,255,255,5,136,0,0,240,65,136,0,0,128,191,136,0,0,128,191,1,185,4,1,2,1,2,3,3,4,4,4]}],[1,{"id":1,"ioInfo":[[["","video"],{"blocking":false,"group":"","name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,23,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,2,255,0,1,0,129,44,1,129,44,1,255,255,255,255,5,136,0,0,240,65,136,0,0,128,191,136,0,0,128,191,1,185,4,1,2,1,2,3,3,4,4,4]}],[2,{"id":2,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,160,67,136,0,0,72,67,185,2,136,0,0,200,67,136,0,0,32,68,136,0,0,180,66,0,136,0,0,128,63,136,0,0,128,63,1,0,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,1,0,0,0,0,134,0,184,11,0,4,0,0,189,0]}],[3,{"id":3,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,160,67,136,0,0,72,67,185,2,136,0,0,200,67,136,0,0,32,68,136,0,0,180,66,0,136,0,0,128,63,136,0,0,128,63,1,0,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,1,0,0,0,0,134,0,184,11,0,4,0,0,189,0]}],[4,{"id":4,"ioInfo":[[["","depth"],{"blocking":false,"group":"","name":"depth","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedLeft"],{"blocking":false,"group":"","name":"rectifiedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedLeft"],{"blocking":false,"group":"","name":"syncedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedRight"],{"blocking":false,"group":"","name":"syncedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedRight"],{"blocking":false,"group":"","name":"rectifiedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","outConfig"],{"blocking":false,"group":"","name":"outConfig","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispCostDump"],{"blocking":false,"group":"","name":"debugDispCostDump","queueSize":8,"type":0,"waitForMessage":false}],[["","confidenceMap"],{"blocking":false,"group":"","name":"confidenceMap","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","disparity"],{"blocking":false,"group":"","name":"disparity","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":4,"type":3,"waitForMessage":false}],[["","left"],{"blocking":false,"group":"","name":"left","queueSize":8,"type":3,"waitForMessage":true}],[["","right"],{"blocking":false,"group":"","name":"right","queueSize":8,"type":3,"waitForMessage":true}]],"name":"StereoDepth","properties":[185,16,185,5,185,9,0,2,136,0,0,122,68,1,0,0,10,3,0,185,7,7,0,185,5,0,2,136,0,0,0,63,0,1,185,4,1,2,136,205,204,204,62,0,185,2,0,134,255,255,0,0,185,2,0,50,185,2,1,0,185,4,255,0,1,0,185,5,1,0,0,50,185,3,0,2,127,185,5,1,128,250,129,244,1,128,250,129,244,1,255,1,0,133,144,1,133,128,2,190,190,1,185,5,189,0,189,0,190,16,16,0,3,255,255,1,0]}],[5,{"id":5,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,0,0,136,0,0,0,0,185,2,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,0,136,0,0,128,63,136,0,0,128,63,0,1,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,0,0,0,0,0,134,0,184,11,0,4,21,33,189,10,97,115,115,101,116,58,109,101,115,104]}],[6,{"id":6,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,99,111,108,111,114,0]}],[7,{"id":7,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,4,108,101,102,116,0]}],[8,{"id":8,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,114,105,103,104,116,0]}],[9,{"id":9,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,100,101,112,116,104,0]}]]}
[2022-11-11 18:35:04.518] [debug] Asset map dump: {"map":{"/node/5/mesh":{"alignment":64,"offset":0,"size":5808}}}
[1844301081DE841200] [1.10.1] [1.319] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B
[1844301081DE841200] [1.10.1] [1.354] [system] [info] ImageManip internal buffer size '379904'B, shave buffer size '35840'B

And the corresponding log when use customized mesh data:

[2022-11-11 18:38:32.867] [debug] Device - OpenVINO version: 2022.1
[2022-11-11 18:38:32.868] [debug] Device - BoardConfig: {"emmc":null,"gpio":[],"logDevicePrints":null,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"maxSpeed":4,"pid":63035,"vid":999},"usb3PhyInternalClock":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null} 
libnop:
0000: b9 0e b9 05 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 b9 02 00 01 ba 00 be be bb 00 bb 00 be be be
0020: be be be be
[2022-11-11 18:38:32.965] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.21.tar.xz' open: 5ms, archive read: 167ms
[2022-11-11 18:38:33.301] [debug] Resources - Archive 'depthai-device-fwp-b3aeaf23ff5857fc8f79d412ceefc08da23e7aad.tar.xz' open: 3ms, archive read: 504ms
[2022-11-11 18:38:34.326] [debug] Searching for booted device: DeviceInfo(name=1.10.1, mxid=1844301081DE841200, X_LINK_BOOTED, X_LINK_USB_VSC, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Memory Usage - DDR: 0.12 / 340.61 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 7.23 / 77.48 MiB, LeonRT Heap: 2.89 / 41.35 MiB
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Temperatures - Average: 39.87 °C, CSS: 41.66 °C, MSS 38.88 °C, UPA: 39.35 °C, DSS: 39.58 °C
[1844301081DE841200] [1.10.1] [1.247] [system] [info] Cpu Usage - LeonOS 75.20%, LeonRT: 1.39%
[2022-11-11 18:38:35.593] [debug] Schema dump: {"connections":[{"node1Id":5,"node1Output":"out","node1OutputGroup":"","node2Id":6,"node2Input":"in","node2InputGroup":""},{"node1Id":3,"node1Output":"out","node1OutputGroup":"","node2Id":5,"node2Input":"inputImage","node2InputGroup":""},{"node1Id":4,"node1Output":"rectifiedRight","node1OutputGroup":"","node2Id":8,"node2Input":"in","node2InputGroup":""},{"node1Id":4,"node1Output":"rectifiedLeft","node1OutputGroup":"","node2Id":7,"node2Input":"in","node2InputGroup":""},{"node1Id":4,"node1Output":"depth","node1OutputGroup":"","node2Id":9,"node2Input":"in","node2InputGroup":""},{"node1Id":3,"node1Output":"out","node1OutputGroup":"","node2Id":4,"node2Input":"right","node2InputGroup":""},{"node1Id":2,"node1Output":"out","node1OutputGroup":"","node2Id":4,"node2Input":"left","node2InputGroup":""},{"node1Id":1,"node1Output":"isp","node1OutputGroup":"","node2Id":3,"node2Input":"inputImage","node2InputGroup":""},{"node1Id":0,"node1Output":"isp","node1OutputGroup":"","node2Id":2,"node2Input":"inputImage","node2InputGroup":""}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","video"],{"blocking":false,"group":"","name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,23,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,1,255,0,1,0,129,44,1,129,44,1,255,255,255,255,5,136,0,0,240,65,136,0,0,128,191,136,0,0,128,191,1,185,4,1,2,1,2,3,3,4,4,4]}],[1,{"id":1,"ioInfo":[[["","video"],{"blocking":false,"group":"","name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":[185,23,185,27,0,3,0,0,0,185,3,0,0,0,185,5,0,0,0,0,0,185,5,0,0,0,0,0,0,0,0,0,0,0,0,185,3,0,0,0,185,3,0,0,0,0,0,0,0,0,0,0,0,0,0,2,255,0,1,0,129,44,1,129,44,1,255,255,255,255,5,136,0,0,240,65,136,0,0,128,191,136,0,0,128,191,1,185,4,1,2,1,2,3,3,4,4,4]}],[2,{"id":2,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,160,67,136,0,0,72,67,185,2,136,0,0,200,67,136,0,0,32,68,136,0,0,180,66,0,136,0,0,128,63,136,0,0,128,63,1,0,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,1,0,0,0,0,134,0,184,11,0,4,0,0,189,0]}],[3,{"id":3,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,160,67,136,0,0,72,67,185,2,136,0,0,200,67,136,0,0,32,68,136,0,0,180,66,0,136,0,0,128,63,136,0,0,128,63,1,0,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,1,0,0,0,0,134,0,184,11,0,4,0,0,189,0]}],[4,{"id":4,"ioInfo":[[["","depth"],{"blocking":false,"group":"","name":"depth","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedLeft"],{"blocking":false,"group":"","name":"rectifiedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedLeft"],{"blocking":false,"group":"","name":"syncedLeft","queueSize":8,"type":0,"waitForMessage":false}],[["","syncedRight"],{"blocking":false,"group":"","name":"syncedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","rectifiedRight"],{"blocking":false,"group":"","name":"rectifiedRight","queueSize":8,"type":0,"waitForMessage":false}],[["","outConfig"],{"blocking":false,"group":"","name":"outConfig","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispCostDump"],{"blocking":false,"group":"","name":"debugDispCostDump","queueSize":8,"type":0,"waitForMessage":false}],[["","confidenceMap"],{"blocking":false,"group":"","name":"confidenceMap","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt2"],{"blocking":false,"group":"","name":"debugDispLrCheckIt2","queueSize":8,"type":0,"waitForMessage":false}],[["","debugDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","debugExtDispLrCheckIt1"],{"blocking":false,"group":"","name":"debugExtDispLrCheckIt1","queueSize":8,"type":0,"waitForMessage":false}],[["","disparity"],{"blocking":false,"group":"","name":"disparity","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","name":"inputConfig","queueSize":4,"type":3,"waitForMessage":false}],[["","left"],{"blocking":false,"group":"","name":"left","queueSize":8,"type":3,"waitForMessage":true}],[["","right"],{"blocking":false,"group":"","name":"right","queueSize":8,"type":3,"waitForMessage":true}]],"name":"StereoDepth","properties":[185,16,185,5,185,9,0,2,136,0,0,122,68,1,0,0,10,3,0,185,7,7,0,185,5,0,2,136,0,0,0,63,0,1,185,4,1,2,136,205,204,204,62,0,185,2,0,134,255,255,0,0,185,2,0,50,185,2,1,0,185,4,255,0,1,0,185,5,1,0,0,50,185,3,0,2,127,185,5,1,128,250,129,244,1,128,250,129,244,1,255,1,0,133,144,1,133,128,2,190,190,1,185,5,189,14,97,115,115,101,116,58,109,101,115,104,76,101,102,116,189,15,97,115,115,101,116,58,109,101,115,104,82,105,103,104,116,129,168,21,20,20,0,3,255,255,1,190]}],[5,{"id":5,"ioInfo":[[["","out"],{"blocking":false,"group":"","name":"out","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":true,"group":"","name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputImage"],{"blocking":true,"group":"","name":"inputImage","queueSize":8,"type":3,"waitForMessage":true}]],"name":"ImageManip","properties":[185,6,185,8,185,7,185,4,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,185,3,185,2,136,0,0,0,0,136,0,0,0,0,185,2,136,0,0,0,0,136,0,0,0,0,136,0,0,0,0,0,136,0,0,128,63,136,0,0,128,63,0,1,185,15,0,0,0,0,0,0,186,0,1,0,186,0,0,0,136,0,0,0,0,0,1,185,3,32,0,0,0,0,0,0,0,134,0,184,11,0,4,21,33,189,10,97,115,115,101,116,58,109,101,115,104]}],[6,{"id":6,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,99,111,108,111,114,0]}],[7,{"id":7,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,4,108,101,102,116,0]}],[8,{"id":8,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,114,105,103,104,116,0]}],[9,{"id":9,"ioInfo":[[["","in"],{"blocking":true,"group":"","name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":[185,3,136,0,0,128,191,189,5,100,101,112,116,104,0]}]]}
[2022-11-11 18:38:35.593] [debug] Asset map dump: {"map":{"/node/4/meshLeft":{"alignment":64,"offset":5824,"size":5544},"/node/4/meshRight":{"alignment":64,"offset":11392,"size":5544},"/node/5/mesh":{"alignment":64,"offset":0,"size":5808}}}
[1844301081DE841200] [1.10.1] [1.318] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B
[1844301081DE841200] [1.10.1] [1.354] [system] [info] ImageManip internal buffer size '379904'B, shave buffer size '35840'B
SzabolcsGergely commented 1 year ago

@developer-mayuan is to possible to share MRE? E.g. your pipeline.