Failed Initialization with Stereo Cameras

[RyanMorris95]

Hello,

I created a custom stereo rig and have been trying to get svo running with it. However, when I run my svo launch file the camera feed never starts up and nothing ever happens, no errors are thrown as well. Is this an issue with my stereo calibration? If I run svo with just one camera everything works fine. The tracking and point cloud looks good. So my best guess is that my extrinsic calibration is off. I have been unable to figure how or why it is wrong though.

Below is my calibration file:

label: ps3eye_forward_stereo
cameras:
- camera:
    label: /sync/camera_left/image_raw
    line-delay-nanoseconds: 0
    image_height: 480
    image_width: 640
    type: pinhole
    intrinsics:
      cols: 1
      rows: 4
      data: [527.304356770005, 528.376891316962, 301.042893012369, 204.555752801064]
    distortion:
      type: radial-tangential
      parameters:
        cols: 1
        rows: 4
        data: [-0.140531817851832, 0.120128496282672, 0., 0.]
  T_B_C:
    cols: 4
    rows: 4
    data: [ 1.0, 0.0, 0.0, 0.0,
            0.0, 1.0, 0.0, 0.0,
            0.0, 0.0, 1.0, 0.0,
            0.0, 0.0, 0.0, 1.0]
- camera:
    label: /sync/camera_right/image_raw
    line-delay-nanoseconds: 0
    image_height: 480
    image_width: 640
    type: pinhole

    intrinsics:
      cols: 1
      rows: 4
      data: [513.739801237591, 514.58311771044, 315.651510398624, 216.259812471907]
    distortion:
      type: radial-tangential
      parameters:
        cols: 1
        rows: 4
        data: [-0.128165242708761, 0.138788351341635, 0., 0.]
  T_B_C:
    cols: 4
    rows: 4
    data: [ 0.999747545636243, 0.0204119840558797, 0.00939126728460886, 0.084301274586288,
            -0.0205963617592674, 0.999588330104035, 0.0199739881372851, 0.000918644251837976,
            -0.0089796924551922, -0.02016237155519, 0.999756392275979, 0.000457187443203893,
            0.0, 0.0, 0.0, 1.0]

Here is my launch file:

<launch>
  <arg name="cam_name" />
  <arg name="calib_file" default="$(find svo_ros)/calib/ps3_stereo_calib.yaml"/>

  <!-- SVO node -->
  <node pkg="svo_ros" type="svo_node" name="svo" clear_params="true" output="screen">

    <!-- Camera topic to subscribe to -->
    <param name="cam0_topic" value="/sync/camera_left/image_raw" type="str" />
    <param name="cam1_topic" value="/sync/camera_right/image_raw" type="str" />

    <!-- Camera calibration file -->
    <param name="calib_file" value="$(arg calib_file)" />

    <!--Parameters-->
    <rosparam file="$(find svo_ros)/param/ps3_stereo2.yaml" />

  </node>

  <!-- RVIZ + SVO GUI -->
  <node name="vis" pkg="rviz" type="rviz" args=" -d $(find svo_ros)/rviz_config.rviz" />
  <node name="svo_gui" pkg="rqt_gui" type="rqt_gui" args="-s rqt_svo.svo.Svo --args --topic svo" />

</launch>

Then I am using the same params as the euroc_stereo file but with use_imu = False.

Please let me know what you think the issue might be.

Thank you for your assistance, Ryan

[RyanMorris95]

The issue was that the stereo camera timestamps weren't exactly the same.

However, I am now getting the issue where SVO never initializes. I believe it is the extrinsic parameters of the camera. I calibrated it will Kalibr and had good reprojections errors in the report so I believe the extrinsics are right, but maybe they aren't in the right format?

Here is my new calibration file and the output of Kalibr:

cameras:
- camera:
    distortion:
      parameters:
        cols: 1
        rows: 4
        data: [-0.0900843007715306, 0.09626027296727115, 0.008889536080228884, 0.013094378840287706]
      type: radial-tangential
    image_height: 480
    image_width: 640
    intrinsics:
      cols: 1
      rows: 4
      data: [536.7420629545321, 536.3834879701145, 330.6999929188717, 249.45529625476476]
    label: cam1
    line-delay-nanoseconds: 0
    type: pinhole
  T_B_C:
    cols: 4
    rows: 4
    data: [0.9996719082596139, 0.005467388424688777, -0.025023658813154506, -0.0835601903296992,
          -0.005079636828296069, 0.9998664554449083, 0.015532822207487014, 0.0013057907775552485,
          0.02510524101211148, -0.015400614917929824, 0.9995661798669836, -0.010201908090312043,
          0.0, 0.0, 0.0, 1.0]
  serial_no: 0
  calib_date: 0
  description: /sync/camera_right/image_raw
- camera:
    distortion:
      parameters:
        cols: 1
        rows: 4
        data: [-0.07616983815394561, 0.10265085592942326, 0.011607702823387219, 0.012889569647631079]
      type: radial-tangential
    image_height: 480
    image_width: 640
    intrinsics:
      cols: 1
      rows: 4
      data: [525.6173676595406, 525.6183780765399, 340.9093020721165, 255.46898552048717]
    label: cam0
    line-delay-nanoseconds: 0
    type: pinhole
  T_B_C:
    cols: 4
    rows: 4
    data: [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0,
      1.0]
  serial_no: 0
  calib_date: 0
  description: /sync/camera_left/image_raw
label: camchain-homeryanCameraCalibration2018-04-02-17-57-09.yaml

Then the output of kalibr:

cam0:
  cam_overlaps: [1]
  camera_model: pinhole
  distortion_coeffs: [-0.07616983815394561, 0.10265085592942326, 0.011607702823387219,
    0.012889569647631079]
  distortion_model: radtan
  intrinsics: [525.6173676595406, 525.6183780765399, 340.9093020721165, 255.46898552048717]
  resolution: [640, 480]
  rostopic: /sync/camera_left/image_raw
cam1:
  T_cn_cnm1:
  - [0.9996719082596139, 0.005467388424688777, -0.025023658813154506, -0.0825601903296992]
  - [-0.005079636828296069, 0.9998664554449083, 0.015532822207487014, 0.0013057907775552485]
  - [0.02510524101211148, -0.015400614917929824, 0.9995661798669836, -0.010201908090312043]
  - [0.0, 0.0, 0.0, 1.0]
  cam_overlaps: [0]
  camera_model: pinhole
  distortion_coeffs: [-0.0900843007715306, 0.09626027296727115, 0.008889536080228884,
    0.013094378840287706]
  distortion_model: radtan
  intrinsics: [536.7420629545321, 536.3834879701145, 330.6999929188717, 249.45529625476476]
  resolution: [640, 480]
  rostopic: /sync/camera_right/image_raw

[sehomi]

Hey Ryan. Did you solve your initialization problem? I am having the same issue with my stereo setup. any help is appreciated!

[lenardxu]

Hey Ryan. I have bumped into the same problem (i.e., "Initialization failed. Not enough triangulated points." ) as yours and sehomis after I did the synchronization of stereo image messages. Have you solved that? Any help is much appreciated!