search

luxonis / depthai

DepthAI Python API utilities, examples, and tutorials.
https://docs.luxonis.com
MIT License
936 stars 232 forks source link

[BUG] OV9782 Variants of POE devices Issue with tracking node #904

Closed justin-larking-pk closed 1 year ago

justin-larking-pk commented 1 year ago

Describe the bug OV9782 variant of the OAK-D-POE-PRO-W does not run pipelines contain the object tracker node. Meanwhile this exact same script and pipeline configuration runs without issue on the OAK-D-POR-POE-W with the imx378 sensor. I have tested this pipeline on four different depthai devices.

As above the OAK-D-PRO-W-POE (OV9782): Does not run halts on the tracker output "q_tracks.get()" OAK-D-PRO-W-DEV: Does not run halts on the object tracker output OAK-D(original variant not s2): Runs without issue OAK-D-PRO-W-POE (imx378): Runs without issue

The only common denominator here seems to be the presence of the ov9782 sensor. In addition I believe its an issue with the tracking node as the script consistently halts on the output of the tracklet queue and other scripts without object tracking run fine on the OAK-D-PRO-W-POE (OV9782).

THere is also a MRE in c++ that can be provided. This was originally tested with depthai-core 2.19.

Minimal Reproducible Example Python MRE: `import depthai as dai import cv2 import numpy as np

rgb_res = "720p"

def main():

pipeline = dai.Pipeline()

spatialdetectionnetwork = pipeline.create(dai.node.MobileNetSpatialDetectionNetwork)
camrgb = pipeline.create(dai.node.ColorCamera)
monoleft = pipeline.create(dai.node.MonoCamera)
monoright = pipeline.create(dai.node.MonoCamera)
depth = pipeline.create(dai.node.StereoDepth)
imu = pipeline.create(dai.node.IMU)
objectTracker = pipeline.create(dai.node.ObjectTracker)
videoenc = pipeline.create(dai.node.VideoEncoder)

xoutdepth = pipeline.create(dai.node.XLinkOut)
xoutvideo = pipeline.create(dai.node.XLinkOut)
xouttracks = pipeline.create(dai.node.XLinkOut)
xoutimu = pipeline.create(dai.node.XLinkOut)
xoutenc = pipeline.create(dai.node.XLinkOut)

xouttracks.setStreamName("tracklets")
xoutdepth.setStreamName("depth")
xoutvideo.setStreamName("video")
xoutimu.setStreamName("imu")
xoutenc.setStreamName("enc")

camrgb.setPreviewSize(300, 300)
camrgb.setInterleaved(False)
camrgb.setColorOrder(dai.ColorCameraProperties.ColorOrder.BGR)
if(rgb_res == "1080p"):
    camrgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)
    camrgb.setIspScale(2, 3)
else:
    camrgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)

camrgb.setPreviewKeepAspectRatio(False)
monoleft.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoleft.setBoardSocket(dai.CameraBoardSocket.LEFT)
monoright.setResolution(dai.MonoCameraProperties.SensorResolution.THE_720_P)
monoright.setBoardSocket(dai.CameraBoardSocket.RIGHT)
depth.setDefaultProfilePreset(dai.node.StereoDepth.PresetMode.HIGH_DENSITY)
depth.initialConfig.setMedianFilter(dai.MedianFilter.KERNEL_7x7)
depth.setLeftRightCheck(True)
depth.setExtendedDisparity(True)
depth.left.setBlocking(False)
depth.right.setBlocking(False)
depth.setDepthAlign(dai.CameraBoardSocket.RGB)
depth.setOutputSize(monoleft.getResolutionWidth(), monoleft.getResolutionHeight())
depth.useHomographyRectification(False)
max_disp = depth.initialConfig.getMaxDisparity()
depth.initialConfig.setConfidenceThreshold(160)
depth.initialConfig.setLeftRightCheckThreshold(10)

config = depth.initialConfig.get()
config.postProcessing.bilateralSigmaValue = 40

spatialdetectionnetwork.setBlobPath("mobilenet-ssd_openvino_2021.2_6shave.blob")
spatialdetectionnetwork.setConfidenceThreshold(0.7)
spatialdetectionnetwork.input.setBlocking(True)
spatialdetectionnetwork.setBoundingBoxScaleFactor(0.5)
spatialdetectionnetwork.setDepthLowerThreshold(100)
spatialdetectionnetwork.setDepthUpperThreshold(10000)

objectTracker.setDetectionLabelsToTrack([15]) 
objectTracker.setTrackerType(dai.TrackerType.ZERO_TERM_COLOR_HISTOGRAM)
objectTracker.setTrackerIdAssignmentPolicy(dai.TrackerIdAssignmentPolicy.SMALLEST_ID)

imu.enableIMUSensor(dai.IMUSensor.GYROSCOPE_RAW, 400)
imu.enableIMUSensor(dai.IMUSensor.ACCELEROMETER_RAW, 500)
imu.setBatchReportThreshold(1)
imu.setMaxBatchReports(1)

camrgb.setFps(15)
monoleft.setFps(15)
monoright.setFps(15)
videoenc.setDefaultProfilePreset(15, dai.VideoEncoderProperties.Profile.H264_MAIN)

camrgb.video.link(xoutvideo.input)
camrgb.preview.link(spatialdetectionnetwork.input)
monoleft.out.link(depth.left)
monoright.out.link(depth.right)

camrgb.video.link(videoenc.input)
videoenc.bitstream.link(xoutenc.input)

objectTracker.out.link(xouttracks.input)
depth.depth.link(xoutdepth.input)
imu.out.link(xoutimu.input)

spatialdetectionnetwork.passthrough.link(objectTracker.inputTrackerFrame)
spatialdetectionnetwork.passthrough.link(objectTracker.inputDetectionFrame)
spatialdetectionnetwork.out.link(objectTracker.inputDetections)
depth.depth.link(spatialdetectionnetwork.inputDepth)

with dai.Device(pipeline) as device:
    q_video = device.getOutputQueue("video", 4, False)
    q_imu = device.getOutputQueue("imu", 4, False)
    q_tracks = device.getOutputQueue("tracklets", 4, False)
    q_depth = device.getOutputQueue("depth", 4, False)
    q_enc = device.getOutputQueue("enc", 4, False)
    count = 0
    while(True):
        count=count+1
        vid_data = q_video.get()
        print("video ", count)
        imu_data = q_imu.get()
        print("imu ", count)
        track = q_tracks.get()
        print("tracks ", count)
        depthdata = q_depth.get()
        print("depth ", count)
        enc_data = q_enc.get()
        print("enc_vid ", count)

        vid_frame = vid_data.getCvFrame()
        print("vid cv2 ", count)
        depth_frame = depthdata.getCvFrame()
        print("depth cv2 ", count)
        enc_frame = enc_data.getData()

        trackletsData = track.tracklets
        imu_packets = imu_data.packets

        cv2.imshow("rgb", vid_frame)
        cv2.imshow("depth", depth_frame)
        key = cv2.waitKey(1)
        if(key == 'q' or key == 'Q'):
            return 0

if name == "main": main()`

Expected behavior This script should open two windows the depth and the rgb frames in two opencv windows. The script is also grabbing the tracklets data from the cameras however nothing is done with it.

Pipeline Graph pipeline

Attach system log

{ "architecture": "64bit ELF", "machine": "x86_64", "platform": "Linux-5.14.0-1055-oem-x86_64-with-glibc2.29", "processor": "x86_64", "python_build": "default Nov 14 2022 12:59:47", "python_compiler": "GCC 9.4.0", "python_implementation": "CPython", "python_version": "3.8.10", "release": "5.14.0-1055-oem", "system": "Linux", "version": "#62-Ubuntu SMP Wed Nov 30 04:54:03 UTC 2022", "win32_ver": "", "packages": [ "absl-py==0.15.0", "actionlib==1.14.0", "addict==2.4.0", "aiohttp==3.8.1", "aiosignal==1.2.0", "angles==1.9.13", "apturl==0.5.2", "argcomplete==1.12.1", "argon2-cffi==21.3.0", "argon2-cffi-bindings==21.2.0", "asttokens==2.0.5", "astunparse==1.6.3", "async-timeout==4.0.2", "attrs==19.3.0", "autobahn==17.10.1", "autocfg==0.0.8", "Automat==0.8.0", "backcall==0.2.0", "base_local_planner==1.17.2", "bcrypt==3.1.7", "beautifulsoup4==4.8.2", "bleach==4.1.0", "blinker==1.4", "blobconverter==1.2.9", "bondpy==1.8.6", "boto3==1.17.39", "botocore==1.20.112", "Brlapi==0.7.0", "cachetools==5.0.0", "camera-calibration==1.17.0", "camera-calibration-parsers==1.12.0", "catkin==0.8.10", "catkin-pkg==0.5.2", "catkin-pkg-modules==0.5.2", "catkin-tools==0.9.2", "cbor==1.0.0", "certifi==2019.11.28", "cffi==1.15.0", "chardet==3.0.4", "charset-normalizer==2.0.12", "Click==7.0", "colorama==0.4.3", "coloredlogs==15.0.1", "command-not-found==0.3", "constantly==15.1.0", "controller-manager==0.19.6", "controller-manager-msgs==0.19.6", "cryptography==2.8", "cupshelpers==1.0", "cv-bridge==1.16.2", "cycler==0.11.0", "Cython==0.29.14", "dbus-python==1.2.16", "debugpy==1.5.1", "decorator==5.1.1", "defer==1.0.6", "defusedxml==0.7.1", "Deprecated==1.2.13", "depthai==2.19.0.0", "depthai-pipeline-graph @ git+https://github.com/geaxgx/depthai_pipeline_graph.git@b5f89bb06b5c421574d991bbdb31f1ebd42118e6", "depthai-sdk==1.9.1", "diagnostic-analysis==1.11.0", "diagnostic-common-diagnostics==1.11.0", "diagnostic-updater==1.11.0", "distinctipy==1.2.2", "distro==1.7.0", "distro-info===0.23ubuntu1", "dnspython==1.16.0", "docker==5.0.3", "docker-compose==1.29.2", "dockerpty==0.4.1", "docopt==0.6.2", "docutils==0.16", "dynamic-reconfigure==1.7.3", "email-validator==1.1.3", "empy==3.3.2", "entrypoints==0.3", "executing==0.8.3", "fake-useragent==0.1.11", "fast_ctc_decode==0.3.0", "fastjsonschema==2.15.3", "fastkml==0.11", "ffmpeg-python==0.2.0", "ffmpy3==0.2.4", "filelock==3.6.0", "fire==0.4.0", "flatbuffers==1.12", "fonttools==4.29.1", "frozenlist==1.3.0", "future==0.18.2", "gast==0.4.0", "gazebo_plugins==2.9.2", "gazebo_ros==2.9.2", "gcloud==0.18.3", "GDAL==3.0.4", "gencpp==0.7.0", "geneus==3.0.0", "genlisp==0.4.18", "genmsg==0.6.0", "gennodejs==2.0.2", "genpy==0.6.15", "geojson==2.5.0", "gluoncv==0.10.5.post0", "google-auth==2.6.2", "google-auth-oauthlib==0.4.6", "google-pasta==0.2.0", "googleapis-common-protos==1.55.0", "googledrivedownloader==0.4", "gpg===1.13.1-unknown", "graphviz==0.8.4", "grpcio==1.34.1", "h5py==3.1.0", "html5lib==1.0.1", "httplib2==0.14.0", "huggingface-hub==0.4.0", "humanfriendly==10.0", "hyperlink==19.0.0", "idna==2.8", "image-geometry==1.16.2", "imagecodecs==2022.2.22", "imageio==2.15.0", "importlib-metadata==1.5.0", "importlib-resources==5.4.0", "imread-from-url==0.1.3", "incremental==16.10.1", "intelhex==2.3.0", "interactive-markers==1.12.0", "ipykernel==6.9.1", "ipython==8.1.1", "ipython-genutils==0.2.0", "ipywidgets==7.6.5", "jedi==0.18.1", "Jinja2==3.0.3", "jmespath==0.10.0", "joblib==1.1.0", "joint-state-publisher==1.15.1", "joint-state-publisher-gui==1.15.1", "jsonschema==4.4.0", "jstyleson==0.0.2", "jupyter-client==7.1.2", "jupyter-core==4.9.2", "jupyterlab-pygments==0.1.2", "jupyterlab-widgets==1.0.2", "jwcrypto==1.0", "kaggle==1.5.12", "keras-nightly==2.5.0.dev2021032900", "Keras-Preprocessing==1.1.2", "keyring==18.0.1", "kiwisolver==1.3.2", "labelImg==1.8.6", "language-selector==0.1", "laser_geometry==1.6.7", "launchpadlib==1.10.13", "lazr.restfulclient==0.14.2", "lazr.uri==1.0.3", "lmdb==1.3.0", "louis==3.12.0", "lxml==4.8.0", "lz4==3.0.2+dfsg", "macaroonbakery==1.3.1", "Markdown==3.1.1", "MarkupSafe==2.1.0", "marshmallow==3.17.0", "matplotlib==3.5.1", "matplotlib-inline==0.1.3", "mbf_abstract_nav==0.4.0", "mcap==0.0.15", "mcap-ros1-support==0.0.8", "message-filters==1.15.15", "mistune==0.8.4", "mixpanel==4.8.3", "mock==3.0.5", "monotonic==1.6", "more-itertools==4.2.0", "mpi4py==3.0.3", "mpmath==1.2.1", "msgpack==0.6.2", "multidict==6.0.2", "mxnet==1.9.1", "nbclient==0.5.11", "nbconvert==6.4.2", "nbformat==5.1.3", "nest-asyncio==1.5.4", "netifaces==0.10.4", "networkx==2.7.1", "nibabel==3.2.2", "nltk==3.7", "nmea_navsat_driver==0.6.1", "nose==1.3.7", "notebook==6.4.8", "numpy==1.24.0", "oauth2client==4.1.3", "oauthlib==3.1.0", "odrive==0.5.4", "olefile==0.46", "onnx==1.11.0", "onnx-simplifier==0.3.7", "onnxoptimizer==0.2.6", "onnxruntime==1.10.0", "onnxruntime-gpu==1.12.1", "open3d==0.10.0.0", "opencv-contrib-python==4.5.5.62", "opencv-python==4.5.5.64", "openvino==2022.1.0", "openvino-dev==2022.1.0", "openvino-telemetry==2022.1.1", "opt-einsum==3.3.0", "osrf-pycommon==2.0.2", "OWSLib==0.19.1", "packaging==21.3", "pandas==1.1.5", "pandocfilters==1.5.0", "panopticapi @ git+https://github.com/cocodataset/panopticapi.git@7bb4655548f98f3fedc07bf37e9040a992b054b0", "paramiko==2.6.0", "parasail==1.2.4", "parso==0.8.3", "pathlib==1.0.1", "pathlib2==2.3.7.post1", "pbr==5.4.5", "pexpect==4.6.0", "pickle-mixin==1.0.2", "pickleshare==0.7.5", "Pillow==9.0.1", "pip==22.3.1", "pkg_resources==0.0.0", "platformdirs==2.5.1", "plotly==4.1.0", "portalocker==2.6.0", "progress==1.6", "prometheus-client==0.13.1", "prompt-toolkit==3.0.28", "protobuf==3.19.4", "psutil==5.5.1", "psycopg2==2.8.4", "ptyprocess==0.7.0", "pure-eval==0.2.2", "py-cpuinfo==8.0.0", "py-ubjson==0.14.0", "pyasn1==0.4.2", "pyasn1-modules==0.2.1", "pycairo==1.16.2", "pyclipper==1.3.0.post2", "pycocotools @ git+https://github.com/philferriere/cocoapi.git@2929bd2ef6b451054755dfd7ceb09278f935f7ad#subdirectory=PythonAPI", "pycparser==2.21", "pycrypto==2.6.1", "pycryptodome==3.14.1", "pycryptodomex==3.6.1", "pycups==1.9.73", "pydantic==1.9.0", "pyDeprecate==0.3.2", "pydicom==2.2.2", "pydot==1.4.1", "pyenchant==2.0.0", "pygeoif==0.7", "Pygments==2.11.2", "PyGObject==3.36.0", "PyHamcrest==1.9.0", "PyJWT==1.7.1", "pymacaroons==0.13.0", "pymongo==3.10.1", "PyNaCl==1.3.0", "pyopencl==2022.1", "PyOpenGL==3.1.0", "pyOpenSSL==19.0.0", "pyparsing==3.0.7", "pypng==0.0.20", "pyproj==2.5.0", "PyQRCode==1.2.1", "PyQt5==5.15.5", "PyQt5-Qt5==5.15.2", "PyQt5-sip==12.11.0", "Pyrebase4==4.5.0", "pyRFC3339==1.1", "pyrsistent==0.18.1", "pyserial==3.4", "python-apt==2.0.0+ubuntu0.20.4.8", "python-can==3.3.2", "python-dateutil==2.8.2", "python-debian===0.1.36ubuntu1", "python-dotenv==0.20.0", "python-gnupg==0.4.5", "python-jwt==3.3.2", "python-qt-binding==0.4.4", "python-slugify==6.1.2", "python-snappy==0.5.3", "pytools==2022.1", "PyTrie==0.2", "pytube==12.1.2", "PyTurboJPEG==1.6.4", "pytz==2021.3", "pyusb==1.2.1", "PyWavelets==1.3.0", "pyxdg==0.26", "PyYAML==6.0", "pyzmq==22.3.0", "qt-dotgraph==0.4.2", "qt-gui==0.4.2", "qt-gui-cpp==0.4.2", "qt-gui-py-common==0.4.2", "Qt.py==1.3.7", "rawpy==0.17.1", "regex==2022.3.15", "reportlab==3.5.34", "requests==2.26.0", "requests-oauthlib==1.3.1", "requests-toolbelt==0.9.1", "requests-unixsocket==0.2.0", "resource_retriever==1.12.7", "retrying==1.3.3", "roman==2.0.0", "rosapi==0.11.16", "rosbag==1.15.15", "rosbags==0.9.11", "rosboost-cfg==1.15.8", "rosbridge-library==0.11.16", "rosbridge-server==0.11.16", "rosclean==1.15.8", "roscreate==1.15.8", "rosdep==0.22.1", "rosdep-modules==0.22.1", "rosdistro==0.9.0", "rosdistro-modules==0.9.0", "rosgraph==1.15.15", "roslaunch==1.15.15", "roslib==1.15.8", "roslint==0.12.0", "roslz4==1.15.15", "rosmake==1.15.8", "rosmaster==1.15.15", "rosmsg==1.15.15", "rosnode==1.15.15", "rosparam==1.15.15", "rospkg==1.4.0", "rospkg-modules==1.4.0", "rospy==1.15.15", "rosservice==1.15.15", "rostest==1.15.15", "rostopic==1.15.15", "rosunit==1.15.8", "roswtf==1.15.15", "rqt-moveit==0.5.10", "rqt-reconfigure==0.5.5", "rqt-robot-dashboard==0.5.8", "rqt-robot-monitor==0.5.14", "rqt-rviz==0.7.0", "rqt_action==0.4.9", "rqt_bag==0.5.1", "rqt_bag_plugins==0.5.1", "rqt_console==0.4.11", "rqt_dep==0.4.12", "rqt_graph==0.4.14", "rqt_gui==0.5.3", "rqt_gui_py==0.5.3", "rqt_image_view==0.4.16", "rqt_launch==0.4.9", "rqt_logger_level==0.4.11", "rqt_msg==0.4.10", "rqt_nav_view==0.5.7", "rqt_plot==0.4.13", "rqt_pose_view==0.5.11", "rqt_publisher==0.4.10", "rqt_py_common==0.5.3", "rqt_py_console==0.4.10", "rqt_robot_steering==0.5.12", "rqt_runtime_monitor==0.5.9", "rqt_service_caller==0.4.10", "rqt_shell==0.4.11", "rqt_srv==0.4.9", "rqt_tf_tree==0.6.3", "rqt_top==0.4.10", "rqt_topic==0.4.13", "rqt_web==0.4.10", "rsa==4.8", "ruamel.yaml==0.17.21", "ruamel.yaml.clib==0.2.6", "rviz==1.14.19", "s3transfer==0.3.7", "sacremoses==0.0.49", "scikit-image==0.19.2", "scikit-learn==0.24.2", "scipy==1.5.4", "screen-resolution-extra==0.0.0", "SecretStorage==2.3.1", "Send2Trash==1.8.0", "sensor-msgs==1.13.1", "sentencepiece==0.1.96", "sentry-sdk==1.5.1", "service-identity==18.1.0", "setuptools==57.4.0", "Shapely==1.8.1.post1", "simplejson==3.16.0", "sip==4.19.21", "six==1.15.0", "smach==2.5.0", "smach-ros==2.5.0", "smbus2==0.4.1", "smclib==1.8.6", "soupsieve==1.9.5", "stack-data==0.2.0", "superannotate==4.3.1", "superannotate-schemas==1.0.40", "sympy==1.11.1", "systemd-python==234", "tensorboard==2.8.0", "tensorboard-data-server==0.6.1", "tensorboard-plugin-wit==1.8.1", "tensorflow==2.5.3", "tensorflow-estimator==2.5.0", "termcolor==1.1.0", "terminado==0.13.2", "testpath==0.6.0", "text-unidecode==1.3", "texttable==1.6.4", "tf==1.13.2", "tf-conversions==1.13.2", "tf2-geometry-msgs==0.7.6", "tf2-kdl==0.7.6", "tf2-py==0.7.6", "tf2-ros==0.7.6", "threadpoolctl==3.1.0", "tifffile==2022.3.25", "tokenizers==0.10.3", "topic-tools==1.15.15", "torch==1.8.1", "torchaudio==0.8.1", "torchmetrics==0.7.2", "torchvision==0.9.1", "tornado==6.1", "tqdm==4.64.0", "traitlets==5.1.1", "transformers==4.16.2", "Twisted==18.9.0", "txaio==2.10.0", "typing-extensions==3.7.4.3", "u-msgpack-python==2.1", "ubuntu-advantage-tools==27.12", "ubuntu-drivers-common==0.0.0", "ufw==0.36", "unattended-upgrades==0.1", "urllib3==1.25.8", "usb-creator==0.3.7", "utm==0.7.0", "wadllib==1.3.3", "wcwidth==0.2.5", "webencodings==0.5.1", "websocket-client==0.59.0", "Werkzeug==2.1.0", "wheel==0.35.1", "widgetsnbextension==3.5.2", "wrapt==1.12.1", "wsaccel==0.6.2", "xacro==1.14.14", "xkit==0.0.0", "xmltodict==0.12.0", "yacs==0.1.8", "yarl==1.7.2", "zipp==3.7.0", "zope.interface==4.7.1", "zstandard==0.18.0" ], "usb": [ { "port": 0, "vendor_id": "0x1d6b", "product_id": "0x0003", "speed": "Super" }, { "port": 8, "vendor_id": "0x0bda", "product_id": "0x0129", "speed": "High" }, { "port": 4, "vendor_id": "0x2357", "product_id": "0x0107", "speed": "High" }, { "port": 14, "vendor_id": "0x045e", "product_id": "0x082e", "speed": "Full" }, { "port": 13, "vendor_id": "0x17ef", "product_id": "0x60a9", "speed": "Full" }, { "port": 12, "vendor_id": "0x03e7", "product_id": "0x2485", "speed": "High" }, { "port": 0, "vendor_id": "0x1d6b", "product_id": "0x0002", "speed": "High" } ], "uname": [ "Linux justin-ThinkStation-P720 5.14.0-1055-oem #62-Ubuntu SMP Wed Nov 30 04:54:03 UTC 2022 x86_64 x86_64" ] }

SzabolcsGergely commented 1 year ago

Hello @justin-larking-pk To run object tracker example checkout latest main, run examples/install_requirements.py and run one of the examples: spatial_object_tracker.py object_tracker.py

They run as it is, tested on OAK-D-W-PRO-POE w/ OV9782 If you want to get rid of the error message modify the following line:

camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_720_P)
justin-larking-pk commented 1 year ago

The spatial_object_tracker.py does indeed run on my OAK-D-W-PRO-POE w/ OV9782 also. I guess this eliminates the object tracker node as being the issue.

However the issue still exists as some OAK-D cameras can run that script I posted above and some cannot.

justin-larking-pk commented 1 year ago

@szabi-luxonis Have you attempted to run my posted example? It may not be that tracking as I thought was breaking it. However this pipeline still only runs on certain depthai cameras, and thus is still an issue.

SzabolcsGergely commented 1 year ago

@justin-larking-pk can you try updating the depthai lib to the latest? We had some bug fixes recently.

justin-larking-pk commented 1 year ago

Updated from depthai on pip 2.19.1 to 2.20.1, no difference still halts on oakd-poe-pro-w (ov9782), continues to run fine on the imx378 variant.

justin-larking-pk commented 1 year ago

Is there no one at Luxonis who can simply run that example that is causing this discrepancy on another ov9782 device and another imx378 one, as a sanity check? I have tried every device I have, the behaviour is exactly the same. The script runs fine on cameras with imx378 sensor, and hatls on ones with the ov9782, this is the only cmmon difference between the cameras I have tested.

SzabolcsGergely commented 1 year ago

The above pipeline works for me @justin-larking-pk :

image

SzabolcsGergely commented 1 year ago

@justin-larking-pk can you set: export DEPTHAI_DEBUG=1 environment variable and post the logs?

justin-larking-pk commented 1 year ago

[2023-01-26 10:46:48.185] [info] DEPTHAI_DEBUG enabled, lowered DEPTHAI_LEVEL to 'debug' [2023-01-26 10:46:48.185] [debug] Python bindings - version: 2.20.1.0 from build: 2023-01-24 16:15:18 +0000 [2023-01-26 10:46:48.185] [debug] Library information - version: 2.20.1, commit: fb873489dba2185f74b4bb95e0ae1b163a386720 from 2023-01-24 16:56:47 +0200, build: 2023-01-24 16:03:12 +0000 [2023-01-26 10:46:48.186] [debug] Initialize - finished [2023-01-26 10:46:48.387] [debug] Resources - Archive 'depthai-bootloader-fwp-0.0.24.tar.xz' open: 9ms, archive read: 192ms [2023-01-26 10:46:48.741] [debug] Device - OpenVINO version: 2022.1 [2023-01-26 10:46:48.742] [debug] Device - BoardConfig: {"camera":[],"emmc":null,"gpio":[],"logDevicePrints":true,"logPath":null,"logSizeMax":null,"logVerbosity":null,"network":{"mtu":0,"xlinkTcpNoDelay":true},"nonExclusiveMode":false,"pcieInternalClock":null,"sysctl":[],"uart":[],"usb":{"flashBootedPid":63037,"flashBootedVid":999,"maxSpeed":4,"pid":63035,"vid":999},"usb3PhyInternalClock":null,"watchdogInitialDelayMs":null,"watchdogTimeoutMs":null} libnop: 0000: b9 10 b9 05 81 e7 03 81 3b f6 81 e7 03 81 3d f6 04 b9 02 00 01 ba 00 be be bb 00 bb 00 be be be 0020: be be be 01 00 bb 00 [2023-01-26 10:46:48.856] [debug] Resources - Archive 'depthai-device-fwp-f888710ca677ca662a4a83103e57c5a1ae2a5c7f.tar.xz' open: 10ms, archive read: 660ms [2023-01-26 10:46:48.902] [debug] Searching for booted device: DeviceInfo(name=192.168.1.129, mxid=194430106195741300, X_LINK_BOOTLOADER, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only [2023-01-26 10:46:48.972] [debug] Connected bootloader version 0.0.19 [2023-01-26 10:46:48.972] [info] New bootloader version available. Device has: 0.0.19, available: 0.0.24 [2023-01-26 10:46:49.828] [debug] Booting FW with Bootloader. Version 0.0.19, Time taken: 856ms [2023-01-26 10:46:49.828] [debug] DeviceBootloader about to be closed... [2023-01-26 10:46:49.829] [debug] XLinkResetRemote of linkId: (0) [2023-01-26 10:46:50.472] [debug] DeviceBootloader closed, 643 [2023-01-26 10:46:50.477] [debug] Searching for booted device: DeviceInfo(name=192.168.1.129, mxid=194430106195741300, X_LINK_BOOTED, X_LINK_TCP_IP, X_LINK_MYRIAD_X, X_LINK_SUCCESS), name used as hint only [194430106195741300] [192.168.1.129] [4.898] [system] [warning] PRINT:LeonCss: GPIO boot mode 0x3, interface SPI_MASTER_EFF Setting aons(0..4) back to boot from flash (offset = 0) ====ENABLE WATCHDOG====1 initial keepalive, countdown: 7 PLL0: 700000 AUX_IO0: 24000 AUX_IO1: 24000 MCFG: 24000 MECFG: 24000 Board init ret 3 eeprom configuration version: 55AA0007 Reading VERSION 7 --- -> eeprom configuration load from user area, status: 0 Reading VERSION 7 --- -> Found a device/board entry matching the eeprom data - Board: OAK-D S2 PoE (generic) Device: Board options: 00000007 --> brdInit ... brdInitAuxDevices: Error: SC = 27: io_initialize expander_cam_gpios_1 [OK]

spi_N25Q_init: Flash JEDEC ID: c2 25 3a SR CR: 40 07 QUAD mode already enabled. Dummy cfg: 0x00 Opening bus for IR driver: 1 Failed to probe IR driver LM3644 Opening bus for IR driver: 2 LM3644 detected, ID = 0x02 ===== IR write bus 2: 0x07 = 0x89 ===== IR write bus 2: 0x02 = 0x01 ===== IR write bus 2: 0x03 = 0x00 ===== IR write bus 2: 0x04 = 0x00 ===== IR write bus 2: 0x05 = 0x00 ===== IR write bus 2: 0x06 = 0x00 ===== IR write bus 2: 0x07 = 0x09 ===== IR write bus 2: 0x08 = 0x1a ===== IR write bus 2: 0x09 = 0x08 ===== IR write bus 2: 0x01 = 0x24 Opening bus for IR driver: 3 Failed to probe IR driver LM3644 Closing EEPROm! MyriaX board configuration pll0 frequency: 700000, ref0 frequency: 24000 Setup network with bootloader config OsDrvCprSysDeviceAction() for PCIe failed: 13 nexus0: pcib0: on nexus0 PCIE_REGS_PCIE_CFG_ADR = 0x60100 PCIE_REGS_PCIE_PLL_CNTRL_ADR = 0x1 pcib0: Link training complete, PCIe downstream link is UP pcib0: myriad_pcie_err_interrupt: SMLH LINK UP pcib0: myriad_pcie_err_interrupt: SMLH LINK DOWN pcib0: myriad_pcie_err_interrupt: RDLH LINK UP pcib0: myriad_pcie_err_interrupt: RDLH LINK DOWN pcib0: myriad_pcie_err_interrupt: CORE RST pcib0: myriad_pcie_err_interrupt: CORE RSTN pcib0: myriad_pcie_err_interrupt: LINK REQ RST pci0: on pcib0 memcpy_dma_init: got DMA id 62 re0: at device 0.0 on pci0 re0: Using Memory Mapping! re0: Using 1 MSI message Overriden invalid mac address, mac: 44:A9:2C:30:F5:9F re0: version:1.95.00 re0: Ethernet address: 0x83d2b908

This product is covered by one or more of the following patents:
US6,570,884, US6,115,776, and US6,327,625. info: re0: Ethernet address: 44:a9:2c:30:f5:9f myriad_mmc2: on nexus0 myriad_mmc2: Memory resource: rid 0 tag 0x0 handle 0x33000000 size 0x1 myriad_mmc2: Found that the memory was NULL - releasing. myriad_mmc2: Error getting voltage regulator resource. myriad_mmc2: Found that the memory was NULL - releasing. myriad_mmc2: Error getting card detect resource. myriad_mmc2: Memory resource: rid 3 tag 0x0 handle 0x20280000 size 0x1 myriad_mmc2: Memory resource: rid 4 tag 0x0 handle 0x83b9c19c size 0x1 myriad_mmc2: IRQ resource: start 0xa001b: irq = 27, priority = 10 myriad_mmc2: Doing host configuration ... 5 myriad_mmc2: CFG 0x0 = 0xffb23840 myriad_mmc2: CFG 0x4 = 0x1c008be myriad_mmc2: CFG 0x24 = 0xb00a621 myriad_mmc2: CFG 0x2c = 0xa000 myriad_mmc2: Polling CFG 0x40 for 0x40 myriad_mmc2: Polling complete, value became 0x40 myriad_mmc2: Base clock = 200000000 Hz myriad_mmc2: Re-tuning mode is 1 myriad_mmc2: Re-Tuning timer count is 1 seconds myriad_mmc2: SDR50 needs tuning. myriad_mmc2: Supports Vdd=3.3 myriad_mmc2: Supports Vdd=3.0 myriad_mmc2: Supports Vdd=1.8 myriad_mmc2: Supports 8-bit bus width myriad_mmc2: Supports High Speed myriad_mmc2: Supports DDR50, DDR52 @ 1.8V myriad_mmc2: Supports HS200 @ 1.8V myriad_mmc2: Supports Driver Type A myriad_mmc2: Supports Driver Type C myriad_mmc2: Supports Driver Type D myriad_mmc2: Data timeout counter value = 0xc myriad_mmc2: Data timeout clock frequency = 48000 kHz, timeout count = 0xc myriad_mmc2: No voltage regulator. myriad_mmc2: Card detect task up! myriad_mmc2: Card attached! mmc0: <MMC/SD bus> on myriad_mmc2 myriad_mmc2: Updating card mode to SD myriad_mmc2: powering off. net.inet.tcp.delayed_ack: 1 -> 0 info: lo0: link state changed to UP ifconfig: inet6: bad value ifcommand setting static ip: 192.168.1.129 mask: 255.255.255.0, gateway: route: bad address: ifcommand failed: exit_code = 68 === Network ready! Is booted from flash by bootloader: 0 Started device discovery service! Waiting for broadcast message.. === Enumerating on socket: Cam_A / RGB / Center

Registered camera CCM OV9782 as /dev/Camera_0 camera socket: 0, name: color config - w: 1280, h: 720, type: MONO config - w: 1280, h: 720, type: COLOR config - w: 1280, h: 800, type: MONO config - w: 1280, h: 800, type: COLOR Adding socket 0: cam 5. Sen name: OV9782 === Enumerating on socket: Cam_B / Left Registered camera TG161B (ov9282) as /dev/Camera_1 camera socket: 1, name: left config - w: 1280, h: 720, type: MONO config - w: 1280, h: 720, type: COLOR config - w: 1280, h: 800, type: MONO config - w: 1280, h: 800, type: COLOR Adding socket 1: cam 5. Sen name: OV9282 === Enumerating on socket: Cam_C / Right Registered camera TG161B (ov9282) as /dev/Camera_2 camera socket: 2, name: right config - w: 1280, h: 720, type: MONO config - w: 1280, h: 720, type: COLOR config - w: 1280, h: 800, type: MONO config - w: 1280, h: 800, type: COLOR Adding socket 2: cam 5. Sen name: OV9282 Initializing XLink... info: re0: link state changed to DOWN initial keepalive, countdown: 6 initial keepalive, countdown: 5 info: re0: link state changed to UP re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC,LINKSTATE> ether 44:a9:2c:30:f5:9f inet 192.168.1.129 netmask 0xffffff00 broadcast 192.168.1.255 media: Ethernet autoselect (1000baseT ) status: active Received device discovery request, sending back - mxid: 194430106195741300, state: 1 Received device discovery request, sending back - mxid: 194430106195741300, state: 1 Done! [194430106195741300] [192.168.1.129] [4.909] [system] [warning] PRINT:LeonCss: Temperature: Driver registered. Temperature: Initialized driver. Temperature: Sensor opened: CSS. Temperature: Sensor opened: MSS. Temperature: Sensor opened: UPA. Temperature: Sensor opened: DSS. [194430106195741300] [192.168.1.129] [4.917] [system] [info] Memory Usage - DDR: 0.12 / 340.42 MiB, CMX: 2.05 / 2.50 MiB, LeonOS Heap: 25.04 / 77.33 MiB, LeonRT Heap: 2.89 / 41.23 MiB [194430106195741300] [192.168.1.129] [4.917] [system] [info] Temperatures - Average: 38.88 °C, CSS: 40.04 °C, MSS 38.65 °C, UPA: 37.71 °C, DSS: 39.11 °C [194430106195741300] [192.168.1.129] [4.917] [system] [info] Cpu Usage - LeonOS 46.86%, LeonRT: 0.59% [194430106195741300] [192.168.1.129] [4.931] [system] [warning] PRINT:LeonCss: I: [Timesync] [ 8897893] [XLin] startSync:130 Timesync | Callback not set [2023-01-26 10:46:55.039] [debug] Asset map dump: {"map":{"/node/0/blob":{"alignment":64,"offset":0,"size":14510848}}} [194430106195741300] [192.168.1.129] [5.052] [system] [warning] PRINT:LeonCss: Calling IPC set assets [194430106195741300] [192.168.1.129] [5.664] [system] [warning] PRINT:LeonCss: Asset: /node/0/blob, pointer: 0x8abb5380, size: 14510848, alignment: 64 [194430106195741300] [192.168.1.129] [5.672] [MonoCamera(3)] [info] Using board socket: 2, id: 2 [194430106195741300] [192.168.1.129] [5.672] [MonoCamera(2)] [info] Using board socket: 1, id: 1 [194430106195741300] [192.168.1.129] [5.674] [ObjectTracker(6)] [info] Init node object tracker [194430106195741300] [192.168.1.129] [5.686] [system] [warning] PRINT:LeonMss: !! The specified ScalingList is not allowed; it will be adjusted!! Calculating resources Number of cmx slices available 16 Reserving slice 16 for object tracker! [194430106195741300] [192.168.1.129] [5.677] [system] [info] SpatialLocationCalculator shave buffer size '29696'B [194430106195741300] [192.168.1.129] [5.678] [system] [info] SIPP (Signal Image Processing Pipeline) internal buffer size '16384'B [194430106195741300] [192.168.1.129] [5.713] [system] [info] ImageManip internal buffer size '258560'B, shave buffer size '34816'B [194430106195741300] [192.168.1.129] [5.713] [system] [info] NeuralNetwork allocated resources: shaves: [0-10] cmx slices: [0-10] ColorCamera allocated resources: no shaves; cmx slices: [12-14] MonoCamera allocated resources: no shaves; cmx slices: [12-14] StereoDepth allocated resources: shaves: [11-11] cmx slices: [11-11] ImageManip allocated resources: shaves: [14-14] no cmx slices. SpatialLocationCalculator allocated resources: shaves: [13-13] no cmx slices. ObjectTracker allocated resources: shaves: [15-15] cmx slices: [15-15]

[194430106195741300] [192.168.1.129] [5.719] [system] [warning] PRINT:LeonMss: sippPalThreadCreate: Thread /SIPP created [194430106195741300] [192.168.1.129] [5.917] [system] [info] Memory Usage - DDR: 14.08 / 340.42 MiB, CMX: 2.40 / 2.50 MiB, LeonOS Heap: 25.33 / 77.33 MiB, LeonRT Heap: 2.97 / 41.23 MiB [194430106195741300] [192.168.1.129] [5.917] [system] [info] Temperatures - Average: 39.64 °C, CSS: 40.74 °C, MSS 39.35 °C, UPA: 39.11 °C, DSS: 39.35 °C [194430106195741300] [192.168.1.129] [5.917] [system] [info] Cpu Usage - LeonOS 68.30%, LeonRT: 6.78% [194430106195741300] [192.168.1.129] [6.049] [system] [warning] PRINT:LeonCss: initial keepalive, countdown: 4 [194430106195741300] [192.168.1.129] [6.269] [system] [warning] PRINT:LeonMss: Initing ImgPreproc system!

Pre-proc Enc mem used: 46080 bytes. Available: 258560 [194430106195741300] [192.168.1.129] [6.280] [system] [warning] PRINT:LeonMss: NeuralNetwork | Available resources: shaveStart: 0, numShaves: 11, mask: 000007FF [194430106195741300] [192.168.1.129] [6.241] [StereoDepth(4)] [info] Using focal length from calibration intrinsics '569.7488' [194430106195741300] [192.168.1.129] [6.263] [StereoDepth(4)] [info] Rectification mesh loaded, will override camera intrinsics/extrinsics matrices [194430106195741300] [192.168.1.129] [6.279] [SpatialDetectionNetwork(0)] [info] Needed resources: shaves: 6, ddr: 7106432 [194430106195741300] [192.168.1.129] [6.279] [SpatialDetectionNetwork(0)] [warning] Network compiled for 6 shaves, maximum available 11, compiling for 5 shaves likely will yield in better performance [194430106195741300] [192.168.1.129] [6.291] [system] [warning] PRINT:LeonMss: finished sippPrePrepareLinesFreeRT [194430106195741300] [192.168.1.129] [6.301] [SpatialDetectionNetwork(0)] [info] Inference thread count: 1, number of shaves allocated per thread: 6, number of Neural Compute Engines (NCE) allocated per thread: 2 [194430106195741300] [192.168.1.129] [6.302] [system] [warning] PRINT:LeonCss: [1970-01-01 00:00:06.293] [info] Time taken to build the pipeline: 625ms [194430106195741300] [192.168.1.129] [6.302] [system] [warning] PRINT:LeonMss: LRT - build pipeline call [194430106195741300] [192.168.1.129] [6.313] [system] [warning] PRINT:LeonCss: == SW-SYNC: 0, cam mask 0x7 !!! Master Slave config is: single_master_slave !!! Starting camera 0 [E] app_guzzi_command_callback():173: command->id:1 [E] app_guzzi_command_callback():193: command "1 0" sent

[194430106195741300] [192.168.1.129] [6.335] [system] [warning] PRINT:LeonCss: [E] iq_debug_create():161: iq_debug address 0x8715e080 [E] hai_cm_driver_load_dtp():852: Features for camera CCM OV9782 are received [E] set_dtp_ids():396: //VIV HAL: Undefined VCM DTP ID 0 [E] set_dtp_ids():405: //VIV HAL: Undefined NVM DTP ID 0 [E] set_dtp_ids():414: //VIV HAL: Undefined lights DTP ID 0 [194430106195741300] [192.168.1.129] [6.357] [system] [warning] PRINT:LeonCss: [E] camera_control_start():347: Camera_id = 0 started.

[E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 [E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 === osDrvOV9782Control: set mode for Camera_0_sen [194430106195741300] [192.168.1.129] [6.379] [system] [warning] PRINT:LeonCss: [E] vpipe_conv_config():1465: Exit Ok [194430106195741300] [192.168.1.129] [6.390] [system] [warning] PRINT:LeonCss: [E] callback():123: Camera CB START_DONE event. Starting camera 1 [E] app_guzzi_command_callback():173: command->id:1 [E] app_guzzi_command_callback():193: command "1 1" sent

[194430106195741300] [192.168.1.129] [6.401] [system] [warning] PRINT:LeonCss: [E] iq_debug_create():161: iq_debug address 0x86c9af00 [194430106195741300] [192.168.1.129] [6.412] [system] [warning] PRINT:LeonCss: [E] hai_cm_driver_load_dtp():852: Features for camera TG161B (ov9282) are received [E] set_dtp_ids():396: //VIV HAL: Undefined VCM DTP ID 0 [E] set_dtp_ids():405: //VIV HAL: Undefined NVM DTP ID 0 [E] set_dtp_ids():414: //VIV HAL: Undefined lights DTP ID 0 [194430106195741300] [192.168.1.129] [6.423] [system] [warning] PRINT:LeonCss: [E] camera_control_start():347: Camera_id = 1 started.

[E] lens_move_to_def_pos_do():251: Error executing af_move_to_pos()! [E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 [E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 === osDrvOV9282Control: set mode for Camera_1_sen inc_camera_process set exposure and gain [194430106195741300] [192.168.1.129] [6.472] [VideoEncoder(7)] [info] Bitrate: auto-set to 1999999 bit/s video 1 [194430106195741300] [192.168.1.129] [6.434] [system] [warning] PRINT:LeonCss: [E] vpipe_conv_config():1465: Exit Ok [194430106195741300] [192.168.1.129] [6.445] [system] [warning] PRINT:LeonCss: [E] callback():123: Camera CB START_DONE event. Starting camera 2 [E] app_guzzi_command_callback():173: command->id:1 [E] app_guzzi_command_callback():193: command "1 2" sent

[194430106195741300] [192.168.1.129] [6.467] [system] [warning] PRINT:LeonCss: [E] iq_debug_create():161: iq_debug address 0x86817c80 [E] hai_cm_driver_load_dtp():852: Features for camera TG161B (ov9282) are received [E] set_dtp_ids():396: //VIV HAL: Undefined VCM DTP ID 0 [E] set_dtp_ids():405: //VIV HAL: Undefined NVM DTP ID 0 [E] set_dtp_ids():414: //VIV HAL: Undefined lights DTP ID 0 [194430106195741300] [192.168.1.129] [6.467] [system] [warning] PRINT:LeonMss: finished sippPrePrepareLinesFreeRT [194430106195741300] [192.168.1.129] [6.478] [system] [warning] PRINT:LeonCss: [E] camera_control_start():347: Camera_id = 2 started.

[E] lens_move_to_def_pos_do():251: Error executing af_move_to_pos()! [E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 [E] hai_cm_sensor_select_mode():164: No suitable sensor mode. Selecting default one - 0 for start 1280x720 at 0x0 fps min 0.000000 max 30.000000 === osDrvOV9282Control: set mode for Camera_2_sen imu 1 [194430106195741300] [192.168.1.129] [6.478] [system] [warning] PRINT:LeonMss: !! The specified ScalingList is not allowed; it will be adjusted!! [194430106195741300] [192.168.1.129] [6.479] [VideoEncoder(7)] [debug] Bitstream frame size: 1499648, num frames in pool: 4 [194430106195741300] [192.168.1.129] [6.489] [system] [warning] PRINT:LeonCss: inc_camera_process set exposure and gain [194430106195741300] [192.168.1.129] [6.511] [system] [warning] PRINT:LeonCss: [E] vpipe_conv_config():1465: Exit Ok [E] guzzi_event_send():324: Send: Event ID=20003 no registered recipient [E] guzzi_event_send():324: Send: Event ID=20004 no registered recipient [194430106195741300] [192.168.1.129] [6.522] [system] [warning] PRINT:LeonCss: [E] callback():123: Camera CB START_DONE event. Starting Guzzi command handling loop... inc_camera_process set exposure and gain [194430106195741300] [192.168.1.129] [6.533] [system] [warning] PRINT:LeonMss: finished sippPrePrepareLinesFreeRT [194430106195741300] [192.168.1.129] [6.566] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! finished sippPrePrepareLinesFreeRT [194430106195741300] [192.168.1.129] [6.632] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [6.698] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [6.764] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [6.830] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [6.896] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [6.918] [system] [info] Memory Usage - DDR: 78.87 / 340.42 MiB, CMX: 2.40 / 2.50 MiB, LeonOS Heap: 66.22 / 77.33 MiB, LeonRT Heap: 6.51 / 41.23 MiB [194430106195741300] [192.168.1.129] [6.918] [system] [info] Temperatures - Average: 41.03 °C, CSS: 42.35 °C, MSS 40.28 °C, UPA: 40.97 °C, DSS: 40.51 °C [194430106195741300] [192.168.1.129] [6.918] [system] [info] Cpu Usage - LeonOS 46.56%, LeonRT: 19.02% [194430106195741300] [192.168.1.129] [6.962] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.028] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.094] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.160] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.226] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.292] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.358] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.424] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.501] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.567] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.633] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.699] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.765] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.831] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.897] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [7.919] [system] [info] Memory Usage - DDR: 78.87 / 340.42 MiB, CMX: 2.40 / 2.50 MiB, LeonOS Heap: 66.22 / 77.33 MiB, LeonRT Heap: 6.51 / 41.23 MiB [194430106195741300] [192.168.1.129] [7.919] [system] [info] Temperatures - Average: 41.49 °C, CSS: 42.35 °C, MSS 40.97 °C, UPA: 41.66 °C, DSS: 40.97 °C [194430106195741300] [192.168.1.129] [7.919] [system] [info] Cpu Usage - LeonOS 44.05%, LeonRT: 1.73% [194430106195741300] [192.168.1.129] [7.963] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.029] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.095] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.161] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.227] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.293] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.359] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.425] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.491] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.557] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.634] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.700] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.766] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.832] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.898] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [8.920] [system] [info] Memory Usage - DDR: 78.87 / 340.42 MiB, CMX: 2.40 / 2.50 MiB, LeonOS Heap: 66.22 / 77.33 MiB, LeonRT Heap: 6.51 / 41.23 MiB [194430106195741300] [192.168.1.129] [8.920] [system] [info] Temperatures - Average: 41.84 °C, CSS: 42.35 °C, MSS 41.89 °C, UPA: 42.12 °C, DSS: 40.97 °C [194430106195741300] [192.168.1.129] [8.920] [system] [info] Cpu Usage - LeonOS 43.61%, LeonRT: 1.72% [194430106195741300] [192.168.1.129] [8.964] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.030] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.096] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.162] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.228] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.294] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.360] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.426] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.492] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.558] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.624] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.690] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.767] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.833] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.899] [system] [warning] PRINT:LeonMss: stereo sync - missing frames! [194430106195741300] [192.168.1.129] [9.922] [system] [info] Memory Usage - DDR: 78.87 / 340.42 MiB, CMX: 2.40 / 2.50 MiB, LeonOS Heap: 66.22 / 77.33 MiB, LeonRT Heap: 6.51 / 41.23 MiB [194430106195741300] [192.168.1.129] [9.922] [system] [info] Temperatures - Average: 42.12 °C, CSS: 42.81 °C, MSS 41.20 °C, UPA: 42.58 °C, DSS: 41.89 °C [194430106195741300] [192.168.1.129] [9.922] [system] [info] Cpu Usage - LeonOS 43.60%, LeonRT: 1.76% [194430106195741300] [192.168.1.129] [9.965] [system] [warning] PRINT:LeonMss: stereo sync - missing frames!

justin-larking-pk commented 1 year ago

I have removed the schema dump that shows the pipeline. Instead i'll give you the pipeline_graph output. pipeline

justin-larking-pk commented 1 year ago

As the full dump is horrible to look at here is a screenshot of log output which contains errors. errors

SzabolcsGergely commented 1 year ago

@justin-larking-pk looks like there's a sync issue between stereo cameras, when 9782 is present. I will be able to look into it later this week.

SzabolcsGergely commented 1 year ago

We found the issue @justin-larking-pk. WIll share the updated FW with the fix soon.

alex-luxonis commented 1 year ago

Fix pushed to the branch release_2.20.2 (https://github.com/luxonis/depthai-core/pull/694) and develop.

justin-larking-pk commented 1 year ago

Awesome, tested the branch and its now working for me, thanks for the help.