Transform data being full of nan, -nan(0x8000000000000) in ROS Noetic(aruco_tracker.cpp)

[real-Sandip-Das]

On debugging, the issue can be narrowed down to the function cv::solvePnP making rvec_final and tvec_final entries full of nan and -nan in void callback_image(...)

Reproducible example: the following launch file that makes use of webcam calibration data saved in ~/.ros/camera_info/camera.yaml

<?xml version="1.0" ?>

<launch>
    <node pkg="cv_camera" name="cv_camera" type="cv_camera_node">
        <param name="device_id" value="0" />
    </node>

    <!-- Run the nodelet manager -->
    <node pkg="nodelet" type="nodelet" name="aruco_tracker_nodelet_manager" args="manager" output="screen"/>

    <node pkg="image_proc" type="image_proc" name="image_proc" ns="/cv_camera" />

    <!-- Load the ArucoTracker nodelet -->
    <node pkg="nodelet" type="nodelet" name="aruco_tracker" args="load aruco_opencv/ArucoTracker aruco_tracker_nodelet_manager" output="screen">
        <param name="cam_base_topic" value="/cv_camera/image_rect"/>
        <param name="marker_size" value="0.2"/>
        <param name="image_is_rectified" value="true"/>
    </node>
</launch>

camera.yaml:

camera_name: "camera"

image_width: 640
image_height: 480

camera_matrix:
  rows: 3
  cols: 3
  data: [933.15867, 0, 657.59, 0, 933.1586, 400.36993, 0, 0, 1]

distortion_model: "plumb_bob"
distortion_coefficients:
  rows: 1
  cols: 5
  data: [0, 0, 0, 0, 0]

rectification_matrix:
  rows: 3
  cols: 3
  data: [1, 0, 0, 0, 1, 0, 0, 0, 1]

projection_matrix:
  rows: 3
  cols: 4
  data: [933.15867, 0, 657.59, 0, 0, 933.1586, 400.36993, 0, 0, 0, 1, 0]

[real-Sandip-Das]

changing the flag in solvePnP to cv::SOLVEPNP_IPPE also results in the same issue

[real-Sandip-Das]

The debug output looks like this:

[real-Sandip-Das]

On changing the flag to cv::SOLVEPNP_P3P, The output rvecs and tvecs are filled with zero, strangely enough

[real-Sandip-Das]

If you are not getting nan in any of your examples, can you please try again with rectified image input?(by e.g. using image_proc)

[real-Sandip-Das]

Upon further investigation, the root cause of the issue seems to be https://github.com/fictionlab/ros_aruco_opencv/issues/46 The issue is resolved(i.e. the output transform data is no longer composed of nan components) in this pull request: https://github.com/fictionlab/ros_aruco_opencv/pull/45 (not yet merged)