/camera_info topic? Using Two Monocular cameras for Stereo?

[javadan]

Hi there

Is the stereo vision only for realsense cameras?

Is there a way to do it with two monocular cameras, with clever remapping?

I currently get stuck because of /camera_info requirements in most of the ISAAC ROS packages.

I've tried dropping a left.yaml and right.yaml (files from the ROS calibration process) under the .ros directory, but the camera_info_url doesn't seem to work.

what(): found unknown ROS arguments: 'camera_info_url=file:///home/admin/.ros/camera_info/left.yaml'

Without calibration, It starts up warning: [WARN] [1650544642.816017010] [argus_monocular]: Cannot get ISyncSensorCalibrationData interface

And /camera_info is all zeroes

admin@chicken:/workspaces/isaac_ros-dev/src/isaac_ros_argus_camera$ ros2 topic echo /camera_info
header:
  stamp:
    sec: 1650544846
    nanosec: 49888246
  frame_id: '6073'
height: 0
width: 0
distortion_model: ''
d: []
k: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
r: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
p: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
binning_x: 0
binning_y: 0
roi:
  x_offset: 0
  y_offset: 0
  height: 0
  width: 0
  do_rectify: false

So something straightforward like the isaac_ros_image_pipeline won't do anything, if I try check the topics.

[ERROR] [1650545104.173691674] [rectify_color]: Rectified topic '/image_rect' requested but camera publishing '/camera_info' is uncalibrated

I'm not sure how to publish the camera info so that the /camera_info topic is usable by the rest of the ISAAC ROS suite. (Or likewise /left/camera_info and /right/camera_info, or whatever, for stereo).

If stereo is possible by remapping a left and a right camera to the input topics required by say, the stereo camera, or the stereo_proc, how would I do that?

Can I remap the output topics on the command line? Or should I be making a launch file to handle the remappings?

Thanks

[hemalshahNV]

Running two instances of Argus monocular node with different .ini files (not YAML) for camera calibration should be fine but the problem will be time synchronizing their frames. You'll also need to introduce the baseline into the camera_info of the right camera to make it work. I can't think of anything that would block you but you probably just want to use a stereo camera (Realsense, ZED, or something else with a rigid housing).

[gordongrigor]

Closing

[gorghino]

Hi @hemalshahNV , could you please pointing out where and how the ini calibrations file must be addressed with latest ISAAC ROS Argus camera? I also tried to edit isaac_ros_argus_camera_mono.launch.py adding camera_info_url as ros-args to argus_mono_container node without success ( I got what(): found unknown ROS arguments: as @javadan )

[hemalshahNV]

camera_info_url is a parameter on the ArgusMonoNode, not an argument for the container (which is a ROS process). You will need to modify the argus_mono_node variable in this launch file and add a parameters field there.

[dakejahl]

@hemalshahNV

Running two instances of Argus monocular node with different .ini files (not YAML) for camera calibration should be fine but the problem will be time synchronizing their frames.

I also would like to use two monocular cameras for stereo vision. I understand that synchronization of the frames is important, can they be synchronized in software using this package? From briefly searching the code it doesn't appear so, but it sounds like libargus can do it. Can you give advice on how I would go about adding frame synchronization?

You'll also need to introduce the baseline into the camera_info of the right camera to make it work

I see the note in the CameraInfo message definition

The first camera always has Tx = Ty = 0. For the right (second) camera of a horizontal stereo pair, Ty = 0 and Tx = -fx' * B, where B is the baseline between the cameras.

How would I modify the camera_info for the right camera? Is it simply changing Tx to *-fx' B** in the below matrix?

       [fx'  0  cx' Tx]
 P =   [ 0  fy' cy' Ty]
       [ 0   0   1   0]