Detection okay with real camera but complete failure with Gazebo

pal-robotics / aruco_ros

Software package and ROS wrappers of the Aruco Augmented Reality marker detector library

MIT License

448 stars 306 forks source link

Detection okay with real camera but complete failure with Gazebo #53

Closed GuillaumeHauss closed 6 years ago

GuillaumeHauss commented 6 years ago

Hi all,

I'm working with Aruco markers to do some localization process. It was working fine and giving me pretty good results so far.

However, I updated Gazebo to 7.12 in order to get some relevant functions on SDF format for frictions models, and now it's a complete failure. The node detects and identifies markers correctly when I use my built-in webcam, but as soon as I feed it a fake image from Gazebo, with a big marker in front of the camera, nothing happened..

Has any of you faced the same issue here? If so, did you find a work around?

Thank you for your help and advice :)

Setup info: Ubuntu 16.04 ROS Kinetic Gazebo 7.12

Cheers Guillaume

v-lopez commented 6 years ago

We actually had some issues with TIAGo and markers that were 8x8cm, and switched to 16x16cm and then it worked perfectly. Ubuntu 16.04, Kinetic and Gazebo 7.12.

Could you post an image of the aruco debug topic?

GuillaumeHauss commented 6 years ago

Hi @v-lopez,

I'm using 5-bits Aruco markers, generated by another ROS package. As I want to see them from really far away (15m), they are huge: 30cm wide.

I'm home now, but I'll post a screen shot of the debug topic first thing tomorrow. From what I saw, markers are perfectly visible in it

GuillaumeHauss commented 6 years ago

As promised, here is a screenshot of the debug topic (left) and the input image from gazebo (right), both visualized using Rviz

I echoed the topic /markers too, to make sure it wasn't a display error, and it's empty. As you can see, thresholding is really efficient, and markers should be detected. I tried to mess around with pixel format from Gazebo and openCV, but ultimately, only one combination works:

L8 in the Gazebo ROS plugin
RGB8 in the ROS node when reading the sensor_msgs::Image (which is weird because it doesn't match the format of the image generated by Gazebo...)
RGB8 again in the publishing of the result image (which is not weird as axes are meant to be displayed in color)
MONO8 in the publishing of the debug image (which is not weird at all as it's only black and white pixels)

I have a monocamera on the car, so I must test the algorithm with grayscale images by the way

GuillaumeHauss commented 6 years ago

Hi,

I found the issue, well my mistake to be correct. I changed the pixel format to RBG8 in the capture of the Image at the beginning of the ROS callback, and somehow that was responsible for the the detector being absolutely inefficient. I changed it back to MONO8 and now it works fine. However, one nice-to-have feature would be a detection of such misalignment in format between Gazebo and ROS :) I'll open another issue to know how I can modify the marker identification function to detect N-bits marker. I've tried to changed the wrapper function but this led to some crappy compilation error :/

I'll close this issue now! have a nice day