Issue when trying to launch yolov8 - using realsense camera

[abs90]

Hello,

I'm trying to launch the yolov8, in 2D and 3D based on the instructions in your readme file. However, I keep facing an issue when I update the "input_image_topic" variable to "/camera/camera/color/image_raw" which matches my realsense topic, can be seen in rviz2.

Please see error below:

~/ros2_ws$ ros2 launch yolov8_bringup yolov8.launch.py [INFO] [launch]: All log files can be found below /home/myaccount/.ros/log/2024-02-04-05-15-45-946799-myaccount-OS-114383 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [yolov8_node-1]: process started with pid [114384] [INFO] [tracking_node-2]: process started with pid [114386] [INFO] [debug_node-3]: process started with pid [114388] [yolov8_node-1] Traceback (most recent call last): [yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node", line 33, in [yolov8_node-1] sys.exit(load_entry_point('yolov8-ros==0.0.0', 'console_scripts', 'yolov8_node')()) [yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 245, in main [yolov8_node-1] rclpy.spin(node) [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/init.py", line 222, in spin [yolov8_node-1] executor.spin_once() [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 713, in spin_once [yolov8_node-1] raise handler.exception() [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/task.py", line 239, in call [yolov8_node-1] self._handler.send(None) [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 418, in handler [yolov8_node-1] await call_coroutine(entity, arg) [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 343, in _execute_subscription [yolov8_node-1] await await_or_execute(sub.callback, msg) [yolov8_node-1] File "/opt/ros/humble/local/lib/python3.10/dist-packages/rclpy/executors.py", line 107, in await_or_execute [yolov8_node-1] return callback(*args) [yolov8_node-1] File "/home/myaccount/ros2_ws/install/yolov8_ros/lib/python3.10/site-packages/yolov8_ros/yolov8_node.py", line 196, in image_cb [yolov8_node-1] results = self.yolo.predict( [yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/engine/model.py", line 250, in predict [yolov8_node-1] self.predictor.setup_model(model=self.model, verbose=is_cli) [yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/engine/predictor.py", line 321, in setup_model [yolov8_node-1] device=select_device(self.args.device, verbose=verbose), [yolov8_node-1] File "/home/myaccount/.local/lib/python3.10/site-packages/ultralytics/utils/torch_utils.py", line 119, in select_device [yolov8_node-1] raise ValueError(f"Invalid CUDA 'device={device}' requested." [yolov8_node-1] ValueError: Invalid CUDA 'device=0' requested. Use 'device=cpu' or pass valid CUDA device(s) if available, i.e. 'device=0' or 'device=0,1,2,3' for Multi-GPU. [yolov8_node-1] [yolov8_node-1] torch.cuda.is_available(): False [yolov8_node-1] torch.cuda.device_count(): 0

[yolov8_node-1] See https://pytorch.org/get-started/locally/ for up-to-date torch install instructions if no CUDA devices are seen by torch. [yolov8_node-1] [ERROR] [yolov8_node-1]: process has died [pid 114384, exit code 1, cmd '/home/myaccount/ros2_ws/install/yolov8_ros/lib/yolov8_ros/yolov8_node --ros-args -r __node:=yolov8_node -r __ns:=/yolo --params-file /tmp/launch_params_315kfwu6 -r image_raw:=/camera/camera/color/image_raw'].

Would appreciate any help. I tried to adjust the QoS but still no Image or data is being shown.

[mgonzs13]

Hey @abs90, that seems a problem with CUDA. Have you installed the Nvidia drivers?

[abs90]

Hey @mgonzs13 appreciate the fast response, that actually makes sense. I tried to use the cpu instead and it seems to work for non-3d yolo8v segmentation.

However, I installed my nvidia drivers, but still seem to face the same problem when launching the 3d yolo8v:

os2 launch yolov8_bringup yolov8_3d.launch.py model:=yolov8m-seg.pt device:=gpu

Will double check my Nvidia installation, any other tips you recommend looking at?

[mgonzs13]

You should check the pytorch installation paying attention to the Nvidia driver supported by your torch's version. You may need to reinstall pytorch. Before running yolo with CUDA, you can check if torch has Nvidia support with the following Python code:

import torch
torch.cuda.is_available()

[abs90]

You should check the pytorch installation paying attention to the Nvidia driver supported by your torch's version. You may need to reinstall pytorch. Before running yolo with CUDA, you can check if torch has Nvidia support with the following Python code:

import torch
torch.cuda.is_available()

That actually seems to have fixed my issue with the Nvidia support and Python Code. Initially it was showing False when I ran the torch.cuda.is_available() command. But I uninstalled pytorch and re-installed, as well as installed cuda-toolkit. It appears that now using the default YOLOv8 model, I no longer need to specify CPU as the driver. However, as soon as I try to run the 3D YOLOv8 model, I don't see any results at all, even though the command executes okay.

I do see this message on my rqt log:

_rqt_topic: Topic "/camera/camera/depth/color/points" has more than one type, choosing the first one of type sensormsgs/msg/Image

No video shows on RVIZ for the 3D detection though.

Please let me know if there's any info that may help to diagnose this?

Thank you for your fast and helpful responses so far!

[mgonzs13]

There are some topics you can use to diagnose. For instance, the /yolo/detections should have the detections of YOLOv8. Echo it to check if the objects are detected. Besides, you can use the rqt_graph to check if all nodes are subscribed to the correct topics.

[abs90]

@mgonzs13, I checked rqt and seems to nodes are subscribed correctly, and match what I see on the rqt graph in your example for the yolov8_3d.

Still investigating why I'm not seeing anything on the image on rviz when running: ros2 launch yolov8_bringup yolov8_3d.launch.py

Also, at least for now I can see that I an use my GPU.

[mgonzs13]

What about the yolo topics? Check them (detections, tracking, detection_3d) to get where is getting stuck.

[abs90]

Seems like I get the tracking, detections but not hte detections_3d, and hence no dbg_image?

[mgonzs13]

So, in this case, it may be getting stuck due to the target_frame. Check if you have this frame. If not, change it to your base frame, for instance, the camera_link, if you are using only a camera.

[mgonzs13]

Hey @abs90, how is this going?