No output or frozen windows with realsense d435

aPonza commented 5 years ago

I'm 90% sure I'm missing something easy but crucial, but I can't get this package to work. What I did:

install realsense2_ros (have had this working for a while now);
install aruco_ros;
built the catkin_ws (at first I was using gcc 7.4.0 but just in case that was the issue I just retried with 5.5.0 to no avail... I only get some warnings in both cases);
roslaunch realsense2_camera rs_rgbd.launch to get the camera up, and these topics/frames:

topics

/camera/aligned_depth_to_color/camera_info
/camera/aligned_depth_to_color/image_raw
/camera/aligned_depth_to_color/image_raw/compressed
/camera/aligned_depth_to_color/image_raw/compressed/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/compressed/parameter_updates
/camera/aligned_depth_to_color/image_raw/compressedDepth
/camera/aligned_depth_to_color/image_raw/compressedDepth/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/compressedDepth/parameter_updates
/camera/aligned_depth_to_color/image_raw/theora
/camera/aligned_depth_to_color/image_raw/theora/parameter_descriptions
/camera/aligned_depth_to_color/image_raw/theora/parameter_updates
/camera/aligned_depth_to_infra1/camera_info
/camera/aligned_depth_to_infra1/image_raw
/camera/aligned_depth_to_infra1/image_raw/compressed
/camera/aligned_depth_to_infra1/image_raw/compressed/parameter_descriptions
/camera/aligned_depth_to_infra1/image_raw/compressed/parameter_updates
/camera/aligned_depth_to_infra1/image_raw/compressedDepth
/camera/aligned_depth_to_infra1/image_raw/compressedDepth/parameter_descriptions
/camera/aligned_depth_to_infra1/image_raw/compressedDepth/parameter_updates
/camera/aligned_depth_to_infra1/image_raw/theora
/camera/aligned_depth_to_infra1/image_raw/theora/parameter_descriptions
/camera/aligned_depth_to_infra1/image_raw/theora/parameter_updates
/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressed/parameter_descriptions
/camera/color/image_raw/compressed/parameter_updates
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/compressedDepth/parameter_descriptions
/camera/color/image_raw/compressedDepth/parameter_updates
/camera/color/image_raw/theora
/camera/color/image_raw/theora/parameter_descriptions
/camera/color/image_raw/theora/parameter_updates
/camera/color/image_rect_color
/camera/color/image_rect_color/compressed
/camera/color/image_rect_color/compressed/parameter_descriptions
/camera/color/image_rect_color/compressed/parameter_updates
/camera/color/image_rect_color/compressedDepth
/camera/color/image_rect_color/compressedDepth/parameter_descriptions
/camera/color/image_rect_color/compressedDepth/parameter_updates
/camera/color/image_rect_color/theora
/camera/color/image_rect_color/theora/parameter_descriptions
/camera/color/image_rect_color/theora/parameter_updates
/camera/color_rectify_color/parameter_descriptions
/camera/color_rectify_color/parameter_updates
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressed/parameter_descriptions
/camera/depth/image_rect_raw/compressed/parameter_updates
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/compressedDepth/parameter_descriptions
/camera/depth/image_rect_raw/compressedDepth/parameter_updates
/camera/depth/image_rect_raw/theora
/camera/depth/image_rect_raw/theora/parameter_descriptions
/camera/depth/image_rect_raw/theora/parameter_updates
/camera/depth_registered/points
/camera/extrinsics/depth_to_color
/camera/extrinsics/depth_to_infra1
/camera/extrinsics/depth_to_infra2
/camera/infra1/camera_info
/camera/infra1/image_rect_raw
/camera/infra1/image_rect_raw/compressed
/camera/infra1/image_rect_raw/compressed/parameter_descriptions
/camera/infra1/image_rect_raw/compressed/parameter_updates
/camera/infra1/image_rect_raw/compressedDepth
/camera/infra1/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra1/image_rect_raw/compressedDepth/parameter_updates
/camera/infra1/image_rect_raw/theora
/camera/infra1/image_rect_raw/theora/parameter_descriptions
/camera/infra1/image_rect_raw/theora/parameter_updates
/camera/infra2/camera_info
/camera/infra2/image_rect_raw
/camera/infra2/image_rect_raw/compressed
/camera/infra2/image_rect_raw/compressed/parameter_descriptions
/camera/infra2/image_rect_raw/compressed/parameter_updates
/camera/infra2/image_rect_raw/compressedDepth
/camera/infra2/image_rect_raw/compressedDepth/parameter_descriptions
/camera/infra2/image_rect_raw/compressedDepth/parameter_updates
/camera/infra2/image_rect_raw/theora
/camera/infra2/image_rect_raw/theora/parameter_descriptions
/camera/infra2/image_rect_raw/theora/parameter_updates
/camera/realsense2_camera_manager/bond
/camera/rgb_camera/parameter_descriptions
/camera/rgb_camera/parameter_updates
/camera/stereo_module/parameter_descriptions
/camera/stereo_module/parameter_updates

frames frames

now then I have tried both roslaunch aruco_ros single.launch markerId:=42 markerSize:=0.07 camera_frame:=camera_color_optical_frame camera_info:=/camera/color/camera_info image:=/camera/color/image_rect_color and roslaunch aruco_ros single.launch markerId:=42 markerSize:=0.07 camera_frame:=camera_link camera_info:=/camera/color/camera_info image:=/camera/color/image_rect_color: in both cases I get these topics up and no errors in the output

topics

/aruco_single/debug
/aruco_single/debug/compressed
/aruco_single/debug/compressed/parameter_descriptions
/aruco_single/debug/compressed/parameter_updates
/aruco_single/debug/compressedDepth
/aruco_single/debug/compressedDepth/parameter_descriptions
/aruco_single/debug/compressedDepth/parameter_updates
/aruco_single/debug/theora
/aruco_single/debug/theora/parameter_descriptions
/aruco_single/debug/theora/parameter_updates
/aruco_single/marker
/aruco_single/parameter_descriptions
/aruco_single/parameter_updates
/aruco_single/pixel
/aruco_single/pose
/aruco_single/position
/aruco_single/result
/aruco_single/result/compressed
/aruco_single/result/compressed/parameter_descriptions
/aruco_single/result/compressed/parameter_updates
/aruco_single/result/compressedDepth
/aruco_single/result/compressedDepth/parameter_descriptions
/aruco_single/result/compressedDepth/parameter_updates
/aruco_single/result/theora
/aruco_single/result/theora/parameter_descriptions
/aruco_single/result/theora/parameter_updates
/aruco_single/transform

and of course I have a print of marker 42 in front of the camera;
if I rosrun image_view image_view image:=/aruco_single/result I get a forever frozen window, and no output on any topic. I'm instead expecting a window identical to the one I'd have by running rosrun image_view image_view image:=/camera/color/image_raw but with colored borders around the marker and a tf frame in the center of the marker, like everybody else's examples show.
if I ./aruco_simple ~/Desktop/image_raw_screenshot_23.04.2019.png where the screenshot is an unmodified screen from the aforementioned rosrun image_view image_view image:=/camera/color/image_raw I get the colored border and number 42 next to it.

Reading through other issues I have found I had an extra opencv installation (2.4) which seemed safe to uninstall, so now I only have the apt packaged ros-kinetic-opencv3:

$ apt-cache policy ros-kinetic-opencv3
ros-kinetic-opencv3:
  Installed: 3.3.1-5xenial-20180809-134307-0800
  Candidate: 3.3.1-5xenial-20180809-134307-0800
  Version table:
 *** 3.3.1-5xenial-20180809-134307-0800 500
        500 http://packages.ros.org/ros/ubuntu xenial/main amd64 Packages
        100 /var/lib/dpkg/status

sudo find / -name "OpenCVConfig.cmake" -print
/opt/ros/kinetic/share/OpenCV-3.3.1-dev/OpenCVConfig.cmake

dpkg -l | grep libopencv
<empty>

I cleaned and rebuilt library and catkin workspace after uninstalling to no avail, again.

How can I debug this situation?

EDIT: roswtf says:

WARNING The following node subscriptions are unconnected:
 * /aruco_single:
   * /stereo/left/camera_info
   * /tf
   * /stereo/left/image_rect_color

so I'll look there for a bit

EDIT2: rewriting the launch file connected the subscriptions, which made the image_view window work, but still with no frames/borders...

aPonza commented 5 years ago

status: the problem lies with ArUco not writing/publishing frames anywhere:

$ roswtf
Loaded plugin tf.tfwtf
No package or stack in context
================================================================================
Static checks summary:

No errors or warnings
================================================================================
Beginning tests of your ROS graph. These may take awhile...
analyzing graph...
... done analyzing graph
running graph rules...
... done running graph rules
running tf checks, this will take a second...
... tf checks complete

Online checks summary:

Found 1 warning(s).
Warnings are things that may be just fine, but are sometimes at fault

WARNING No tf messages

I'm trying to go step by step but it's not helping much, I'll also double check again the Debugging tf tutorial for good measure

aPonza commented 5 years ago

The problem seemed to be no marker was ever detected (maybe the marker I printed is unsupported from this repo? I have number 42 from the default ARUCO_MIP_36h12 dictionary).

Incorporating #44 seems to solve the issue.

Spj-Zhao commented 2 years ago

@aPonza Hi, I am in the same situation as you，I also get a forever frozen window, and no output on any topic.How did you solve it？And where is the aruco_simple file?

pal-robotics / aruco_ros

No output or frozen windows with realsense d435 #66