search

pal-robotics / aruco_ros

Software package and ROS wrappers of the Aruco Augmented Reality marker detector library
MIT License
448 stars 306 forks source link

Monocular camera calibration #59

Open GianlucaCerilli opened 5 years ago

GianlucaCerilli commented 5 years ago

Good evening,

I have a monocular camera mounted on the end-effector of a kinova manipulator and I am trying to use Aruco_ros to detect the marker for getting the camera extrinsic parameters witih Ros Kinetic. In the documentation I see how to use it with a reem gazebo simulator and a stereo camera.

Can I ask you the commands to run it without simulator and with a mono camera? I tried but when I run rosrun image_view image_view image:=/aruco_single/result It does not work.

Thank you very much

awesomebytes commented 5 years ago

You just need to make your own launchfile like: https://github.com/pal-robotics/aruco_ros/blob/kinetic-devel/aruco_ros/launch/single.launch

With your data (markerid you printed, real life size of it, camera info topic, image topic). Note that to have accurate results your camera should be calibrated intrinsically (so what's published in the camera_info topic should be correct, or you provide the rectified topic already, like in the example launchfile). You can ignore the stereo parameters as you may see in the launchfile they are there for convenience for the stereo case.

GianlucaCerilli commented 5 years ago

HI @awesomebytes ,

Thank you for your explanation. So, in my case should I have something like this?

But I am not sure on what to insert in marker_frame, ref_frame and camera_frame fields in my case since I am not in the simulator and I still have to extrinsically calibrate the camera. By the way, when I connect the camera I have these topics:

/pylon_camera_node/camera_info /pylon_camera_node/grab_images_raw/cancel /pylon_camera_node/grab_images_raw/feedback /pylon_camera_node/grab_images_raw/goal /pylon_camera_node/grab_images_raw/result /pylon_camera_node/grab_images_raw/status /pylon_camera_node/image_raw /pylon_camera_node/image_raw/compressed /pylon_camera_node/image_raw/compressed/parameter_descriptions /pylon_camera_node/image_raw/compressed/parameter_updates /pylon_camera_node/image_raw/compressedDepth /pylon_camera_node/image_raw/compressedDepth/parameter_descriptions /pylon_camera_node/image_raw/compressedDepth/parameter_updates /pylon_camera_node/image_raw/theora /pylon_camera_node/image_raw/theora/parameter_descriptions /pylon_camera_node/image_raw/theora/parameter_updates

Thank you very much again!

awesomebytes commented 5 years ago

Hey @GianlucaCerilli . That looks like a correct launchfile to me. You did set image_is_rectified to True and I don't really believe that is true because you'd have a topic called image_rect or similar. If you set it to False the node itself will use the calibration published in /pylon_camera_node/camera_info to rectify the image for you internally. Which, you really wanna calibrate your camera intrinsically, following the ros monocular camera calibration tutorial. Otherwise your 3D poses detected will be off by some amount (depending on your camera) so your extrinsic calibration will be even more off.

camera_frame is the frame of your camera, as the name suggests, for doing the internal frame transformations. That's the one you have in your URDF where the camera is positioned. For example if your robot has arm_link_7 -> end_effector_frame -> pylon_camera_frame as where the camera is, you want to set it to pylon_camera_frame.

marker_frame is the name of the frame that will be published in TF where the detection of the aruco marker happens. Useful for doing transformations in the relative pose to the marker. aruco_marker_frame is a good default.

ref_frame is frame in which the marker pose will be referred, so if for your use-case it's more useful to have the marker Pose topic published in, say, base_link, just set that there.

GianlucaCerilli commented 5 years ago

Hi @GianlucaCerilli ,

Thank you for your explanation. Now all the extrinsic calibration works. But in the end, the final camera pose that I get from the extrinsic calibration (aruco + visp) seems to be incorrect on the Z axis and I think the problem it is related to the result that I get from aruco. Indeed, when I run aruco, the estimate that I get on the Z direction is not really accurate (some centimeters of error). It seems to compute the distance from the marker to the lens of the camera instead of computing it from the marker to the sensor of the camera. So, there are some centimeters of errors in that direction.

How can I do the get a more accurate result in this sense?

Thank you very much

awesomebytes commented 5 years ago

Well you should add as camera_frame the frame of your camera, I mean, the optical frame, which is probably the sensor of your camera, just behind the lens. If you think that is the error.

In many cameras there is the camera_link > camera_optical_frame. Maybe you need to add that to your model to fix the issue? (If the error you are seeing is constant, of course).

GianlucaCerilli commented 5 years ago

Hi @awesomebytes ,

Thank you for your reply. Where can I check if there is camera_optical_frame for my camera or if I am using the correct one?

From rostopic echo I see only these topics:

/pylon_camera_node/camera_info /pylon_camera_node/grab_images_raw/cancel /pylon_camera_node/grab_images_raw/feedback /pylon_camera_node/grab_images_raw/goal /pylon_camera_node/grab_images_raw/result /pylon_camera_node/grab_images_raw/status /pylon_camera_node/grab_images_rect/cancel /pylon_camera_node/grab_images_rect/feedback /pylon_camera_node/grab_images_rect/goal /pylon_camera_node/grab_images_rect/result /pylon_camera_node/grab_images_rect/status /pylon_camera_node/image_raw /pylon_camera_node/image_raw/compressed /pylon_camera_node/image_raw/compressedDepth /pylon_camera_node/image_raw/compressedDepth/parameter_descriptions /pylon_camera_node/image_raw/compressedDepth/parameter_updates /pylon_camera_node/image_raw/compressed/parameter_descriptions /pylon_camera_node/image_raw/compressed/parameter_updates /pylon_camera_node/image_raw/theora /pylon_camera_node/image_raw/theora/parameter_descriptions /pylon_camera_node/image_raw/theora/parameter_updates /pylon_camera_node/image_rect

GianlucaCerilli commented 5 years ago

I have also noticed that the frame of aruco marker is not really correct:

image

Even if it looks wrong only in Rviz, since I get:

camera->marker translation: x: -0.0326698213869 y: -0.0221858542217 z: 0.474173845014 rotation: x: 0.28700230383 y: -0.628464615532 z: 0.636957014297 w: -0.341976119867

LiGengLei commented 5 years ago

Hello, @GianlucaCerilli When I use the aruco to estimate the position of something, I have the same question: the distance of z axis has too many error(about 1.5 times the true distance). How do you solve this question?Thank you.

LiGengLei commented 5 years ago

Update: The program itself is error-free, the problem is that I set the resolution of the camera incorrectly.

sngweicong commented 5 years ago

Hi, @LiGengLei,

I am facing the exact same problem, where the distance of z is just wrong. May I know where do you set the resolution of the camera?