Occlusion robustness

[AlexReimann]

We did some occlusion tests of an tagStandard41h12 tag and the results were actually terrible.

Are other tags / tag families more robust to occlusion? The original paper shows some highly occluded but still working tags:

https://april.eecs.umich.edu/media/pdfs/olson2011tags.pdf

Here is a bag of our test:

https://energyrobotics-my.sharepoint.com/:u:/g/personal/alexander_reimann_energy-robotics_com/EZvgFJxKYiNHp1mj_XxDT-sBZv6rWAcsGvo2LIOaHa3fFg?e=s2bD8t

The camera has some exposure switching turned on to cover more lightning conditions. And we know it's kind of a crappy camera, but it's real world application after all.

Settings used for detection:

tag_family:        'tagStandard41h12'
tag_threads:       2

# detection of quads can be done on a lower-resolution image,
# improving speed at a cost of pose accuracy and a slight
# decrease in detection rate. Decoding the binary payload is
# still done at full resolution. .
tag_decimate:      1.0        # default: 1.0

# What Gaussian blur should be applied to the segmented image
# (used for quad detection?)  Parameter is the standard deviation
# in pixels.  Very noisy images benefit from non-zero values
# (e.g. 0.8).
tag_blur:         0.8       # default: 0.0

# When non-zero, the edges of the each quad are adjusted to "snap
# to" strong gradients nearby. This is useful when decimation is
# employed, as it can increase the quality of the initial quad
# estimate substantially. Generally recommended to be on (1).
# Very computationally inexpensive. Option is ignored if
# tag_decimate = 1.
tag_refine_edges:  1          # default: 1

# When non-zero, write a variety of debugging images to the
# current working directory at various stages through the
# detection process. (Somewhat slow).
tag_debug:         0          # default: 0

max_hamming_dist:  2          # default: 2 # high values are computational intensive

[christian-rauch]

Your data is quite low resolution. Did you try higher resolutions?

Other than that, I can only say that testing with a single tag family is not really exhaustive :-) Did you test other tag families? Did you try to reproduce the results from the paper?

[s-trinh]

None of AprilTag 2, AprilTag 3 are robust to occlusion. Only the first version is. This is stated in the AprilTag 2 paper:

The original AprilTag detector used image gradients to detect high-contrast edges. This has the advantage of being robust to shadows and variations in lighting over previous methods which used naive thresholding. Detection of partially occluded tags was made possible by first fitting segments to the gradients, then searching over combinations of segments that formed four-sided shapes, or quads. A disadvantage of the segment-first approach is the large number of candidate quads that are generated.

The improvements to the tag detector were inspired by user feedback about common use cases. We learned that in most deployments, detection of partially occluded tags is of limited utility. Occluded tags often have one or more bit errors, and most users disable decoding of tags with bit errors due to the impact on false positive rates. No known users accept tags with more than two bit errors, which enables a faster decode algorithm. In our experience, the increased detection speed is a favorable tradeoff against the ability to recover partially occluded tag borders.

[christian-rauch]

@AlexReimann I am going to close this. If you need occlusion robustness, you can switch to an earlier version of the library.

[AlexReimann]

Thank you very much for the info :+1:

I actually tested with a different camera and the results there were a bit better:

[GuidoBartoli]

@AlexReimann I am going to close this. If you need occlusion robustness, you can switch to an earlier version of the library.

We are evaluating AprilTags for outdoor robot localization, can you recommend the most occlusion-resilient variant? (Tag36h11, TagStandard, TagCircle, ...)