chili-epfl / chilitags

Robust Fiducial Markers for Augmented Reality And Robotics
http://chili.epfl.ch/software
123 stars 57 forks source link

add reprojection-based precision benchmark #77

Closed qbonnard closed 6 months ago

qbonnard commented 9 years ago

This is a work in progress to discuss a reprojection based benchmark to measure the precision of the location of the tags

qbonnard commented 9 years ago

@ayberkozgur wrote:

Ok, after looking at your branch, I finally understand what you mean by reprojection test. But in that code, Chilitags seems to be doing nothing? Anyway, I think this could be useful somehow (in parallel with your test):

  1. Generate artificial tag image, know exact corner locations
  2. Pass image to Chilitags3D<float>, then reproject back using projection matrix
  3. Compare exact corner locations with backprojected corner locations

Note that above, Chilitags3D<double> doesn't make sense because if float is close enough to actual values, double will certainly be (unless float is masking a crazy one in a billion bug).

qbonnard commented 9 years ago

Sorry about the total lack of documentation on this code. Actually the goal of the reprojection is to have some kind of groundtruth on non artificial images. The idea is to make a good estimation of a tag configuration, and then measure the noise on individual tags. This is the simplest implementation, where the tag configuration is a regular grid (with id's coding the position of the grid). solvePnP is used to find a precise position/pose of this grid, and then each tag is projected using the estimation. Then we measure the distance between the reprojection and the original detection of the tag.

Ideally, on an artificial image of a grid, the distances should be 0. I'm surprised to see that there is already some errors... I haven't experimented with solvePnP parameters yet to determine whether it can be improved, or if there is an intrinsic lack of precision in chilitags.

However, if a tag is indeed out of place, like in the screenshot-offset image, we do measure a bigger average error.

Is that more clear ?