search

luxonis / depthai-ros

Official ROS Driver for DepthAI Sensors.
MIT License
259 stars 189 forks source link

Unable to set required resolution on OAK-1 Lite camera #393

Open makarandmandolkar opened 1 year ago

makarandmandolkar commented 1 year ago

Describe the bug We are trying to use the OAK-1 Lite camera for our application. The website specifies 4K resolutions and 4208x3120 (13MP), but its been a little trouble achieving the same. We are using depthai-ros noetic build from source, and launching the camera.launch from the depthai_ros_driver package. It uses the camera.yaml file for the arguments. On changing line 46 in the camera.yaml "rgb_i_resolution: '4K'", it was observed that it will still result in 2K output. Moreover, any other value set in the line 46 will result in HD output or error. Currently setting that value at 13MP results in a resolution of 2806X2080 and not 4208x3120. Adjusting other values regarding the resolution in the config files didn't prove very helpful either.

Can you help/guide on how to achieve 4K, 13MP and HD resolution, just by changing the values in the config files?

Screenshots image image

Pipeline Graph

{"connections":[{"node1Id":2,"node1Output":"out","node1OutputGroup":"","node2Id":0,"node2Input":"inputControl","node2InputGroup":""},{"node1Id":0,"node1Output":"isp","node1OutputGroup":"","node2Id":1,"node2Input":"in","node2InputGroup":""}],"globalProperties":{"calibData":null,"cameraTuningBlobSize":null,"cameraTuningBlobUri":"","leonCssFrequencyHz":700000000.0,"leonMssFrequencyHz":700000000.0,"pipelineName":null,"pipelineVersion":null,"sippBufferSize":18432,"sippDmaBufferSize":16384,"xlinkChunkSize":-1},"nodes":[[0,{"id":0,"ioInfo":[[["","video"],{"blocking":false,"group":"","id":10,"name":"video","queueSize":8,"type":0,"waitForMessage":false}],[["","still"],{"blocking":false,"group":"","id":8,"name":"still","queueSize":8,"type":0,"waitForMessage":false}],[["","isp"],{"blocking":false,"group":"","id":7,"name":"isp","queueSize":8,"type":0,"waitForMessage":false}],[["","preview"],{"blocking":false,"group":"","id":9,"name":"preview","queueSize":8,"type":0,"waitForMessage":false}],[["","raw"],{"blocking":false,"group":"","id":6,"name":"raw","queueSize":8,"type":0,"waitForMessage":false}],[["","frameEvent"],{"blocking":false,"group":"","id":5,"name":"frameEvent","queueSize":8,"type":0,"waitForMessage":false}],[["","inputConfig"],{"blocking":false,"group":"","id":4,"name":"inputConfig","queueSize":8,"type":3,"waitForMessage":false}],[["","inputControl"],{"blocking":true,"group":"","id":3,"name":"inputControl","queueSize":8,"type":3,"waitForMessage":false}]],"name":"ColorCamera","properties":{"boardSocket":0,"cameraName":"","colorOrder":0,"fp16":false,"fps":30.0,"imageOrientation":-1,"initialControl":{"aeLockMode":false,"aeRegion":{"height":0,"priority":0,"width":0,"x":0,"y":0},"afRegion":{"height":0,"priority":0,"width":0,"x":0,"y":0},"antiBandingMode":0,"autoFocusMode":3,"awbLockMode":false,"awbMode":0,"brightness":0,"chromaDenoise":0,"cmdMask":0,"contrast":0,"effectMode":0,"expCompensation":0,"expManual":{"exposureTimeUs":0,"frameDurationUs":0,"sensitivityIso":0},"frameSyncMode":0,"lensPosAutoInfinity":0,"lensPosAutoMacro":0,"lensPosition":0,"lowPowerNumFramesBurst":0,"lowPowerNumFramesDiscard":0,"lumaDenoise":0,"saturation":0,"sceneMode":0,"sharpness":0,"strobeConfig":{"activeLevel":0,"enable":0,"gpioNumber":0},"strobeTimings":{"durationUs":0,"exposureBeginOffsetUs":0,"exposureEndOffsetUs":0},"wbColorTemp":0},"interleaved":false,"isp3aFps":0,"ispScale":{"horizDenominator":3,"horizNumerator":2,"vertDenominator":3,"vertNumerator":2},"numFramesPoolIsp":3,"numFramesPoolPreview":4,"numFramesPoolRaw":3,"numFramesPoolStill":4,"numFramesPoolVideo":4,"previewHeight":300,"previewKeepAspectRatio":true,"previewWidth":300,"rawPacked":null,"resolution":3,"sensorCropX":-1.0,"sensorCropY":-1.0,"stillHeight":-1,"stillWidth":-1,"videoHeight":2806,"videoWidth":2080}}],[1,{"id":1,"ioInfo":[[["","in"],{"blocking":true,"group":"","id":2,"name":"in","queueSize":8,"type":3,"waitForMessage":true}]],"name":"XLinkOut","properties":{"maxFpsLimit":-1.0,"metadataOnly":false,"streamName":"rgb_isp"}}],[2,{"id":2,"ioInfo":[[["","out"],{"blocking":false,"group":"","id":1,"name":"out","queueSize":8,"type":0,"waitForMessage":false}]],"name":"XLinkIn","properties":{"maxDataSize":5242880,"numFrames":8,"streamName":"rgb_control"}}]]}

Attach system log { "architecture": "64bit ELF", "machine": "x86_64", "platform": "Linux-5.15.0-78-generic-x86_64-with-glibc2.29", "processor": "x86_64", "python_build": "default May 26 2023 14:05:08", "python_compiler": "GCC 9.4.0", "python_implementation": "CPython", "python_version": "3.8.10", "release": "5.15.0-78-generic", "system": "Linux", "version": "#85~20.04.1-Ubuntu SMP Mon Jul 17 09:42:39 UTC 2023", "win32_ver": "", "packages": [ "action-tutorials-py==0.14.4", "actionlib==1.14.0", "actionlib-tools==1.14.0", "aiohttp==3.8.4", "aiosignal==1.3.1", "ament-cmake-test==1.1.6", "ament-copyright==0.10.7", "ament-cppcheck==0.10.7", "ament-cpplint==0.10.7", "ament-flake8==0.10.7", "ament-index-python==1.2.0", "ament-lint==0.10.7", "ament-lint-cmake==0.10.7", "ament-package==0.13.1", "ament-pep257==0.10.7", "ament-uncrustify==0.10.7", "ament-xmllint==0.10.7", "angles==1.9.13", "aniso8601==7.0.0", "argcomplete==1.12.3", "async-timeout==4.0.2", "base-local-planner==1.17.3", "beeply==1.0.2", "beepy==1.0.7", "bondpy==1.8.6", "camera-calibration==1.17.0", "camera-calibration-parsers==1.12.0", "catkin==0.8.10", "cffi==1.15.1", "charset-normalizer==3.1.0", "click==8.1.3", "clickclick==20.10.2", "cmake==3.26.4", "connexion==2.6.0", "controller-manager==0.19.6", "controller-manager-msgs==0.19.6", "cv-bridge==1.16.2", "deepspeech==0.9.3", "demo-nodes-py==0.14.4", "diagnostic-analysis==1.11.0", "diagnostic-common-diagnostics==1.11.0", "diagnostic-updater==1.11.0", "domain-coordinator==0.9.2", "dynamic-reconfigure==1.7.3", "examples-rclpy-executors==0.11.2", "examples-rclpy-minimal-action-client==0.11.2", "examples-rclpy-minimal-action-server==0.11.2", "examples-rclpy-minimal-client==0.11.2", "examples-rclpy-minimal-publisher==0.11.2", "examples-rclpy-minimal-service==0.11.2", "examples-rclpy-minimal-subscriber==0.11.2", "ffmpeg-python==0.2.0", "filelock==3.12.2", "flake8-blind-except==0.2.0", "flake8-builtins==1.5.3", "flake8-class-newline==1.6.0", "flake8-comprehensions==3.8.0", "flake8-deprecated==1.3", "flake8-docstrings==1.6.0", "flake8-import-order==0.18.1", "flake8-quotes==3.3.1", "Flask==2.2.3", "Flask-GraphQL==2.0.1", "frozenlist==1.3.3", "gencpp==0.7.0", "geneus==3.0.0", "genlisp==0.4.18", "genmsg==0.6.0", "gennodejs==2.0.2", "genpy==0.6.15", "graphene==2.1.9", "graphql-core==2.3.2", "graphql-relay==2.0.1", "graphql-server-core==1.2.0", "grepros==0.5.0", "heic-to-jpg==0.1.8", "image-geometry==1.16.2", "importlib-metadata==4.11.1", "importlib-resources==5.12.0", "inflection==0.5.1", "iniconfig==1.1.1", "install==1.3.5", "interactive-markers==1.12.0", "iso8601==1.0.2", "itsdangerous==2.1.2", "Jinja2==3.1.2", "joint-state-publisher==1.15.1", "joint-state-publisher-gui==1.15.1", "jsonschema==4.17.3", "jsonschema-spec==0.1.4", "laser-geometry==1.6.7", "launch==0.17.2", "launch-ros==0.14.5", "launch-testing==0.17.2", "launch-testing-ros==0.14.5", "launch-xml==0.17.2", "launch-yaml==0.17.2", "lazy-object-proxy==1.9.0", "lit==16.0.6", "llvmlite==0.40.1", "MarkupSafe==2.1.2", "message-filters==1.16.0", "mpmath==1.3.0", "multidict==6.0.4", "nav2-common==1.0.12", "neo4j==5.9.0", "networkx==3.1", "numba==0.57.1", "numpy==1.21.0", "nvidia-cublas-cu11==11.10.3.66", "nvidia-cuda-cupti-cu11==11.7.101", "nvidia-cuda-nvrtc-cu11==11.7.99", "nvidia-cuda-runtime-cu11==11.7.99", "nvidia-cudnn-cu11==8.5.0.96", "nvidia-cufft-cu11==10.9.0.58", "nvidia-curand-cu11==10.2.10.91", "nvidia-cusolver-cu11==11.4.0.1", "nvidia-cusparse-cu11==11.7.4.91", "nvidia-nccl-cu11==2.14.3", "nvidia-nvtx-cu11==11.7.91", "openai==0.27.8", "openai-whisper==20230314", "openapi-schema-validator==0.4.4", "openapi-spec-validator==0.3.1", "opencv-contrib-python==4.7.0.72", "opencv-python==4.8.0.76", "osrf-pycommon==2.0.0", "pandas==1.5.2", "pathable==0.4.3", "pbkdf2==1.3", "pdfkit==1.0.0", "Pillow==9.4.0", "Pillow-SIMD==9.0.0.post1", "pkgutil-resolve-name==1.3.10", "pocketsphinx==5.0.1", "promise==2.3", "pvleopard==1.2.2", "pvleoparddemo==1.2.2", "pvporcupine==2.2.1", "pvporcupinedemo==2.2.1", "pvrecorder==1.1.1", "py==1.10.0", "PyAudio==0.2.13", "pycparser==2.21", "pyheif==0.7.1", "pyrsistent==0.19.3", "pyserial==3.5", "PySimpleGUI==4.60.1", "pysine==0.9.2", "pytest==7.0.1", "pytest-repeat==0.9.1", "pytest-rerunfailures==10.2", "python-dateutil==2.8.2", "python-qt-binding==0.4.4", "pyttsx3==2.90", "pytz==2022.7", "qt-dotgraph==0.4.2", "qt-gui==0.4.2", "qt-gui-cpp==0.4.2", "qt-gui-py-common==0.4.2", "quality-of-service-demo-py==0.14.4", "rclpy==1.9.3", "rcutils==4.0.4", "regex==2023.6.3", "requests==2.31.0", "resource-retriever==1.12.7", "rfc3339-validator==0.1.4", "ros2-controllers-test-nodes==1.5.1", "ros2action==0.13.5", "ros2bag==0.9.2", "ros2cli==0.13.5", "ros2component==0.13.5", "ros2controlcli==1.6.0", "ros2doctor==0.13.5", "ros2interface==0.13.5", "ros2launch==0.14.5", "ros2lifecycle==0.13.5", "ros2multicast==0.13.5", "ros2node==0.13.5", "ros2param==0.13.5", "ros2pkg==0.13.5", "ros2run==0.13.5", "ros2service==0.13.5", "ros2topic==0.13.5", "rosapi==0.11.16", "rosbag==1.16.0", "rosbag2-py==0.9.2", "rosboost-cfg==1.15.8", "rosbridge-library==0.11.16", "rosbridge-server==0.11.16", "rosclean==1.15.8", "roscreate==1.15.8", "rosgraph==1.16.0", "rosidl-adapter==2.2.3", "rosidl-cli==2.2.3", "rosidl-cmake==2.2.3", "rosidl-generator-c==2.2.3", "rosidl-generator-cpp==2.2.3", "rosidl-generator-py==0.11.4", "rosidl-parser==2.2.3", "rosidl-runtime-py==0.9.1", "rosidl-typesupport-c==1.2.1", "rosidl-typesupport-cpp==1.2.1", "rosidl-typesupport-fastrtps-c==1.2.1", "rosidl-typesupport-fastrtps-cpp==1.2.1", "rosidl-typesupport-introspection-c==2.2.3", "rosidl-typesupport-introspection-cpp==2.2.3", "roslaunch==1.16.0", "roslib==1.15.8", "roslibpy==1.3.0", "roslint==0.12.0", "roslz4==1.16.0", "rosmake==1.15.8", "rosmaster==1.16.0", "rosmsg==1.16.0", "rosnode==1.16.0", "rosparam==1.16.0", "rospy==1.16.0", "rosservice==1.16.0", "rostest==1.16.0", "rostopic==1.16.0", "rosunit==1.15.8", "roswtf==1.16.0", "rpyutils==0.2.0", "rqt==1.1.2", "rqt-action==0.4.9", "rqt-bag==0.5.1", "rqt-bag-plugins==0.5.1", "rqt-console==0.4.11", "rqt-dep==0.4.12", "rqt-graph==0.4.14", "rqt-gui==0.5.3", "rqt-gui-py==0.5.3", "rqt-image-view==0.4.17", "rqt-launch==0.4.9", "rqt-logger-level==0.4.11", "rqt-moveit==0.5.10", "rqt-msg==0.4.10", "rqt-nav-view==0.5.7", "rqt-plot==0.4.13", "rqt-pose-view==0.5.11", "rqt-publisher==0.4.10", "rqt-py-common==0.5.3", "rqt-py-console==0.4.10", "rqt-reconfigure==0.5.5", "rqt-robot-dashboard==0.5.8", "rqt-robot-monitor==0.5.14", "rqt-robot-steering==0.5.12", "rqt-runtime-monitor==0.5.9", "rqt-rviz==0.7.0", "rqt-service-caller==0.4.10", "rqt-shell==0.4.11", "rqt-srv==0.4.9", "rqt-tf-tree==0.6.3", "rqt-top==0.4.10", "rqt-topic==0.4.13", "rqt-web==0.4.10", "rviz==1.14.20", "Rx==1.6.3", "scapy==2.5.0", "scripts==3.5.2", "semantic-version==2.10.0", "sensor-msgs==1.13.1", "setuptools==68.0.0", "setuptools-rust==1.6.0", "simpleaudio==1.0.4", "smach==2.5.2", "smach-ros==2.5.2", "smclib==1.8.6", "sounddevice==0.4.6", "SpeechRecognition==3.10.0", "sros2==0.10.3", "stag==1.2.1", "sympy==1.12", "tabulate==0.8.10", "teleop-twist-keyboard==2.3.2", "tf==1.13.2", "tf-conversions==1.13.2", "tf2-geometry-msgs==0.7.6", "tf2-kdl==0.7.6", "tf2-py==0.7.6", "tf2-ros==0.7.6", "tf2-ros-py==0.17.5", "tf2-tools==0.17.5", "tiktoken==0.3.1", "toml==0.10.2", "tomli==2.0.1", "topic-monitor==0.14.4", "topic-tools==1.16.0", "torch==2.0.1", "tqdm==4.65.0", "triton==2.0.0", "turtlebot3-example==2.1.2", "turtlebot3-teleop==2.1.2", "typing-extensions==4.5.0", "urdfdom-py==1.1.0", "Werkzeug==2.2.3", "wifi==0.3.8", "xacro==1.14.16", "yarl==1.9.2", "zipp==3.15.0" ], "usb": [ "NoLib" ], "uname": [ "Linux makarand 5.15.0-78-generic #85~20.04.1-Ubuntu SMP Mon Jul 17 09:42:39 UTC 2023 x86_64 x86_64" ] } System info gathered successfully - saved as "log_system_information.json"

Serafadam commented 1 year ago

Hi, thanks for the verbose information. Could you check out if newest Noetic version still causes issues?

makarandmandolkar commented 1 year ago

Thank you for your quick reply. The launch file crashes, if I don't make the following change camera_i_pipeline_type: RGB. Also, the image appears inverted. Can you please guide how to set the required resolution from the camera.yaml only, as I see the camera.yaml file has been changed in the latest change?

Serafadam commented 1 year ago

Hi, would it be possible for you to test this PR and see if things work better? Regarding parameter files, removing rgb_i_width and rgb_i_height parameters and setting following should be enough: rgb_i_resolution: 13MP rgb_i_set_isp_scale: false

makarandmandolkar commented 1 year ago

Hi, thanks for the update. I did try the PR, it looks great. There are a few observations though, like I was able to get the 13MP working with the said resolution. However, when I visualized the image topic in Rviz, it still appeared inverted. Please let me know if you need more info. Looking forward to hearing from your side.

image

Serafadam commented 1 year ago

Hi, does the image also appear inverted when running the demo script?. In general, you can use rgb_i_sensor_img_orientation: VERTICAL_FLIP

rahulsharma11 commented 1 year ago

Hi, would it be possible for you to test this PR and see if things work better? Regarding parameter files, removing rgb_i_width and rgb_i_height parameters and setting following should be enough: rgb_i_resolution: 13MP rgb_i_set_isp_scale: false

Hi. I ran the same thing in Jetson Orin nano. I am getting 6fps on that. Anything can be done to get more fps with same resolution?

Thanks.

Serafadam commented 1 year ago

Hi @rahulsharma11 , could you share your setup and resource usage? Are you subscribing to images over the network or on the device?

rahulsharma11 commented 1 year ago

Hi @Serafadam .

I am using Jetson Orin Nano platform. Over there i am able to get 6fps -> 12MP 9fps -> 4k.

I also tried intel NUC i7, over there i am getting 12fps -> 12MP 16fps -> 4k

I have directly connected the cameras over USB type C and subscribing the topics directly on the board.

Serafadam commented 1 year ago

Hi, I assume this is using ROS2 Humble or Noetic? There are several factors to consider here:

rahulsharma11 commented 1 year ago

Hi. I am using ROS Noetic.

And it is not only showing in rostopic hz, i am dumping the data also and over there i can confirm the topic counts which are at 6fps(12MP) and 10fps(4k).

rahulsharma11 commented 1 year ago

Hi, does the image also appear inverted when running the demo script?. In general, you can use rgb_i_sensor_img_orientation: VERTICAL_FLIP

I found setting the parameter as AUTO is making the feed correct. VERTICAL_FLIP is making the image correct but with horizontal mirror.

Serafadam commented 1 year ago

Hi, newest version will have AUTO by default. Also, regarding Noetic frequency, I noticed high one core load increase when using rostopic hz. I suspect that the issue could be from the rostopic tool as it is not loaded as a nodelet so it cannot use shared memory transport (which could make a difference when subscribing to large messages).

rahulsharma11 commented 1 year ago

Thanks for rostopic node insight. But i am dumping the data with rosbag record and i can find the topic count according to 6FPS(12MP) over there. That means my USB only is limiting the data dump? i am using typec to typec in Thunderbolt port, so i am doubting if USB is limiting the data dump.

Serafadam commented 1 year ago

According to this issue, rosbag also doesn't support nodelet mechanism. USB speed should be printed out in logs after starting the camera. One additional tool to inspect the performance could be the depthai pipeline graph. To use it,

pip install git+https://github.com/luxonis/depthai_pipeline_graph.git
pip install depthai
pipeline_graph run "roslaunch depthai_ros_driver camera.launch"

This should output FPS of different nodes on a camera.

rahulsharma11 commented 1 year ago

ok. Thanks for the suggestion. I will look into it.

Furthermore if dumping with rosbag record is also not good way to dump. If you can suggest few options to dump the data which is faster and good way, i can follow that too.

Thanks for your support.

Serafadam commented 1 year ago

When it comes to other approaches, maybe using something like Foxglove and foxglove_bridge could work better in this case

rahulsharma11 commented 1 year ago

I can see foxglove is very interactive tool for real time data visualisation, i think i can't use it to record the published topics.