Oak-D not working

[dmorris0]

OS: Ubuntu Distro: Humble, Installed on RPI, Humble on my laptop

I want to see the images from the OAK-D lite on my Turtlebot4 Lite. But they are not available in rqt image-viewer plugin, and I see errors from the OAK-D Lite when I run the diagnostics package:

ros2 launch turtlebot4_viz view_diagnostics.launch.py

I flashed the RPI image a couple times and keep getting this error. I connected the Oak-D to my Windows PC, and the demo runs run.

Not sure what to do about this. Help!

[hilary-luo]

Is your robot docked? There is a power saving feature which disables the oakd while docked.

[dmorris0]

No, it is not docked. But I found out that despite this error message, I can still subscribe to and view the topic: /oakd/rgb/preview/image_raw So I think that means the Oak-D camera is working after all.

[hilary-luo]

@dmorris0 thanks for you patience, I have been busy. It does sound like the camera is working after all, I will look into why the diagnostics is suggesting otherwise.

[dmorris0]

Well, I'm not sure if everything is working. In the Turtlebot4 manual on the Oak-D sensor (https://turtlebot.github.io/turtlebot4-user-manual/software/sensors.html#oak-d) it says to run this command: ros2 launch turtlebot4_bringup oakd.launch.py I do that and get a namespace error -- I don't have any namespace for my bot. Then I try ros2 launch turtlebot4_bringup oakd.launch.py namespace:=/ And it runs. However, I don't get any output and have no way to tell what it is supposed to do.

I think there needs to be better and more description for how to use the Oak-D sensor and how to see the output, which I'm not convinced there is any aside from the preview color image.

[Akalpita30]

@dmorris0 i ran into same problem with my turtlebot4. is there any chance that you found the solution to this issue. i would really appreciate help on this.

[hilary-luo]

@dmorris0 @Akalpita30 it seems like that diagnostic tool is outdated and is looking for the wrong topics so I am going to be looking into that and updating it.

@dmorris0 You should not be launching the camera directly for a standard image, it is launched automatically, just undock your robot. Relaunching this same node will cause issues.

If you are looking to use the stereo data, currently you will need to build from source and use the ros2 launch turtlebot4_bringup oakd_example.launch.py and then modify the camera.yaml to configure the settings that you are looking for.

I agree that the documentation on this topic is lacking and it is one of the things on my to do list to improve that. I also want to give the option directly in the setup configuration whether or not to launch the camera as stereo. If you guys end up going through this before I get there, feel free to make PR's both to the code and to the documentation.

[dmorris0]

Thanks for the update @hilary-luo. Is there any way that the camera can publish the full color image in compressed format, rather than the tiny preview image in raw? Perhaps that is possible if building from source? Can you tell me what exactly needs to be built from source on the Raspberry Pi?

[dmorris0]

I guess you mean build the turtlebot4 repo. I can clone this into a workspace on the Raspberry Pi, but it's not clear how this workspace get called on boot-up.

[dmorris0]

Ah, I found it -- the bringup package is in turtlebot4_robot. Well, I cloned and built this repo in a ROS workspace. But if run the example, I get an error that the sensor is already in use, which I suppose it is. Not sure how to get around that since at boot time the sensor gets started.

$ ros2 launch turtlebot4_bringup oakd_example.launch.py
[INFO] [launch]: All log files can be found below /home/ubuntu/.ros/log/2023-11-07-21-09-52-007874-turtlebot4-15-4645
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [rgb_stereo_node-1]: process started with pid [4656]
[rgb_stereo_node-1] terminate called after throwing an instance of 'std::runtime_error'
[rgb_stereo_node-1]   what():  Failed to connect to device, error message: X_LINK_DEVICE_ALREADY_IN_USE
[ERROR] [rgb_stereo_node-1]: process has died [pid 4656, exit code -6, cmd '/opt/ros/humble/lib/depthai_examples/rgb_stereo_node --ros-args --params-file /tmp/launch_params__ilb66kd --params-file /tmp/launch_params_m3rw55v3 --params-file /tmp/launch_params_6s2c4k3q --params-file /tmp/launch_params_6xzvqlem --params-file /tmp/launch_params_c506pqfu --params-file /tmp/launch_params_gg1eyps4 --params-file /tmp/launch_params_5ewayj2i --params-file /tmp/launch_params_fhw2yoz4 --params-file /tmp/launch_params_yoxx1kdg --params-file /tmp/launch_params_f8jox5vk --params-file /tmp/launch_params_663xk51p --params-file /tmp/launch_params_hp0y5xff --params-file /tmp/launch_params_myns1bzu --params-file /tmp/launch_params_kfjrrghs'].

[hilary-luo]

@dmorris0 I was referring to building turtlebot4_robot from source. This package contains the turtlebot4_bringup. When the robot boots up it automatically runs the turtlebot4 service which was installed using the turtlebot4_setup package. When you are testing things you can stop the turtlebot4 service and then when you are ready you can reinstall the service to link to your local build. So the steps that you need to take are:

1. Clone, source /opt/ros/humble/setup.bash and build turtlebot4_robot package
2. Modify the oakd_pro.yaml (or oakd_lite.yaml for Turtlebot 4 Lite) file to enable the stereo topic, see examples of different config files here: https://github.com/luxonis/depthai-ros/tree/humble/depthai_ros_driver/config
3. Link to your custom workspace using the turtlebot4-setup tool:
   • run turtlebot4-setup
   • navigate to ROS Setup -> Bash Setup
   • select WORKSPACE_SETUP
   • enter the path to your local setup.bash for your workspace. This will allow the turtlebot4 setup tool to locate and source these packages. e.g. /home/administrator/ros_ws/install/setup.bash
4. Reinstall the turtlebot4 service using the turtlebot4-setup tool under Robot Upstart -> Install.
   • run turtlebot4-setup
   • navigate to ROS Setup -> Robot Upstart
   • select Install and agree as necessary
5. Now your robot should automatically run your local build whenever it is turned on.

The last few steps are using the turtlebot4-setup tool. The full usage of this tool is documented here: https://turtlebot.github.io/turtlebot4-user-manual/software/turtlebot4_setup.html#configuration-tools

As a note I am not currently able to test those instructions so they are to the best of my memory at the moment.

If you want to launch things directly for testing then you can also use the turtlebot4-setup tool under robot upstart (or equivalent bash command) to stop the service. This will stop everything that is running in the background so you can launch things in the foreground manually. This is useful for troubleshooting. If you are launching the robot manually you can either launch the oakd launch file directly (ros2 launch turtlebot4_bringup oakd.launch.py) or launch all robot systems (ros2 launch turtlebot4_bringup standard.launch.py or lite if using Turtlebot4 lite)

[hilary-luo]

This issue is being closed due to inactivity. If you are still experiencing the issue, feel free to reopen this ticket when you are ready to continue the troubleshooting process.

[AncientGrey]

Hi, is it possible to provide a camera (oak-d pro) config .yaml file, where every Sensor is included and activated? That way it is easier to get an overview of all the possible uses of the camera and the turtlebot.

[nielmistry]

This would be great ^