luxonis / depthai-ros

Official ROS Driver for DepthAI Sensors.
MIT License
255 stars 186 forks source link

OAK-1-MAX non stereo / non spatial YOLO example node #262

Closed MAVProxyUser closed 1 year ago

MAVProxyUser commented 1 year ago

Tested successfully with OAK-1-MAX, OAK-D-S2, OAK-D-PRO. Original OpenCV AI Kit: Oak-1 version failed with following error:

[yolov4_image_node-1]   what():  There is no Camera data available corresponding to the the requested cameraId

This is likely due to "Device type:" being reported as blank https://github.com/luxonis/depthai-ros/issues/257#issuecomment-1480627256

[component_container-1] [INFO] [1679549436.807773594] [oak]: Device type: 

To test with other cams launch with:

ros2 launch depthai_examples yolov4_publisher_nodepth.launch.py camera_model:=OAK-1-MAX

Query YOLO hits with:

ubuntu@ubuntu:/tmp/depthai-ros$ ros2 topic list 
/color/camera_info
/color/image
/color/yolov4_Img_detections
/joint_states
/parameter_events
/robot_description
/rosout
/tf
/tf_static
ubuntu@ubuntu:/tmp/depthai-ros$ ros2 topic hz /color/yolov4_Img_detections
average rate: 19.158
    min: 0.002s max: 0.109s std dev: 0.04062s window: 21
average rate: 19.203
    min: 0.002s max: 0.109s std dev: 0.04038s window: 42
average rate: 19.233
    min: 0.002s max: 0.109s std dev: 0.03964s window: 62
average rate: 19.447
    min: 0.002s max: 0.109s std dev: 0.03927s window: 83
ubuntu@ubuntu:/tmp/depthai-ros$ ros2 topic echo /color/yolov4_Img_detections
---
header:
  stamp:
    sec: 1679631788
    nanosec: 407148768
  frame_id: oak_rgb_camera_optical_frame
detections: []
---
header:
  stamp:
    sec: 1679631788
    nanosec: 457638565
  frame_id: oak_rgb_camera_optical_frame
detections:
- header:
    stamp:
      sec: 0
      nanosec: 0
    frame_id: ''
  results:
  - hypothesis:
      class_id: '63'
      score: 0.5892834663391113
    pose:
      pose:
        position:
          x: 0.0
          y: 0.0
          z: 0.0
        orientation:
          x: 0.0
          y: 0.0
          z: 0.0
          w: 1.0
      covariance:
      - 0.0
      - 0.0
      - 0.0
      - 0.0
...
  bbox:
    center:
      position:
        x: 73.0
        y: 306.0
      theta: 0.0
    size_x: 148.0
    size_y: 238.0
  id: '63'
---
Serafadam commented 1 year ago

Hi, thanks a lot!, minor comment, I'd change it to use the same launch file as current yolo launch, and just add argument to choose if we want spatial version or not.

MAVProxyUser commented 1 year ago

Cool @Serafadam it was a stretch for me to get this sorted out. I may need to rely on you for that last mile. I'm not super familiar with launch file semantics.

MAVProxyUser commented 1 year ago

@Serafadam in your opinion what would be the best way to handle the small differences? If camera_model is OAK-1 then launch the other node? That seem reasonable?

ubuntu@ubuntu:~/depthai-ros$  diff depthai_examples/launch/yolov4_publisher_nodepth.launch.py depthai_examples/launch/yolov4_publisher.launch.py 
22c22
<     camera_model = LaunchConfiguration('camera_model',  default = 'OAK-1-MAX')
---
>     camera_model = LaunchConfiguration('camera_model',  default = 'OAK-D')
35a36
>     subpixel           = LaunchConfiguration('subpixel',          default = True)
38a40,41
>     lrCheckTresh       = LaunchConfiguration('lrCheckTresh',      default = 5)
>     monoResolution     = LaunchConfiguration('monoResolution',  default = '400p')
43c46
<         description='The model of the camera. Using a wrong camera model can disable camera features. Valid models: `OAK-1, OAK-1-MAX`.')
---
>         description='The model of the camera. Using a wrong camera model can disable camera features. Valid models: `OAK-D, OAK-D-LITE`.')
99a103,107
>     declare_subpixel_cmd = DeclareLaunchArgument(
>         'subpixel',
>         default_value=subpixel,
>         description='Enables subpixel stereo detection.')
> 
114a123,132
>     declare_lrCheckTresh_cmd = DeclareLaunchArgument(
>         'lrCheckTresh',
>         default_value=lrCheckTresh,
>         description='LR Threshold is the threshod of how much off the disparity on the l->r and r->l  ')
> 
>     declare_monoResolution_cmd = DeclareLaunchArgument(
>         'monoResolution',
>         default_value=monoResolution,
>         description='Contains the resolution of the Mono Cameras. Available resolutions are 800p, 720p & 400p for OAK-D & 480p for OAK-D-Lite.')
>   
129,130c147,148
<     yolov4_image_node = launch_ros.actions.Node(
<             package='depthai_examples', executable='yolov4_image_node',
---
>     yolov4_spatial_node = launch_ros.actions.Node(
>             package='depthai_examples', executable='yolov4_spatial_node',
136c154,155
<                         {'resourceBaseFolder': resourceBaseFolder}])
---
>                         {'resourceBaseFolder': resourceBaseFolder},
>                         {'monoResolution': monoResolution}])
157a177
>     ld.add_action(declare_subpixel_cmd)
160a181,182
>     ld.add_action(declare_lrCheckTresh_cmd)
>     ld.add_action(declare_monoResolution_cmd)
162c184
<     ld.add_action(yolov4_image_node)
---
>     ld.add_action(yolov4_spatial_node)
Serafadam commented 1 year ago

Instead of device check, I'd add a spatial flag that launches given node, in cpp we can probe device how much sensors it has and return error if it only has one sensor (that's basically how it works now in depthai_ros_driver)

MAVProxyUser commented 1 year ago

Ok @Serafadam I did my best! Sadly I had to move off my humble install for compatibility with other devices. I'm on noetic for the time being. Perhaps you can see if I got it right!?

Serafadam commented 1 year ago

LGTM, could you change target branche to humble?

MAVProxyUser commented 1 year ago

LGTM, could you change target branche to humble?

I think I tried to... but now it says "This branch has conflicts that must be resolved". Do you know how to handle that?

MAVProxyUser commented 1 year ago

@Serafadam do you need me to rebase, or do something else with this?

Serafadam commented 1 year ago

Hi @MAVProxyUser, sorry for the delay, it seems that you either need to pull changes from humble branch and resolve confilcts. You could also try creating a new branch from humble and apply those changes here, that's probably a simpler way. I can also do that after my current backlog clears up a little bit.

MAVProxyUser commented 1 year ago

This look any better @Serafadam ?