Help with thresholding parameters

[naiveHobo]

Describe the bug In my use case, I come across several such instances where there are odd lighting conditions and/or shadows on the tags in the environment. Looking at the debug images, this particular tag seems to be thresholding well and there's a decent enough cluster around the tag. However, this tag is never detected as a candidate quad. I'm using this image as a general example but there are several instances where I come across such situations. What can be done to make this situation better?

To Reproduce Steps to reproduce the behavior: Run apriltag_demo on the attached input image with the following parameters:

td->nthreads = 1;
td->quad_decimate = 0.0;
td->quad_sigma = 0.0;

td->qtp.max_nmaxima = 10;
td->qtp.min_cluster_pixels = 5;

td->qtp.max_line_fit_mse = 10.0;
td->qtp.cos_critical_rad = cos(10 * M_PI / 180);
td->qtp.deglitch = false;
td->qtp.min_white_black_diff = 5;

td->refine_edges = true;
td->decode_sharpening = 0.25;

Expected behavior At least a quad to be detected around the tag in the image, decoding is not as big an issue.

Input Image

Screenshots

debug_threshold

debug_segmentation

debug_clusters

debug_quads_raw

Operating Sytem Ubuntu 20.04

Installation Method I built AprilTag from source following the instructions in the README

Code version 3.3.0

[christian-rauch]

I guess that this is essentially the same issue as https://github.com/AprilRobotics/apriltag/issues/161 and https://github.com/AprilRobotics/apriltag/issues/235. To cite https://github.com/AprilRobotics/apriltag/issues/161#issuecomment-881967591:

The reason the tag on the right is not detected is because of the sharp lighting gradient on the bottom left corner. This causes the adaptive thresholding (see debug_threshold.pnm) to classify a chunk of the white border as black, which causes the derived edge (see debug_clusters.pnm) to loop all the way back around the outside of the tag.

Without changing the thresholding implementation in the apriltag library you have to preprocess the image to better separate the black/white content of the tag.

You may also get lucky with decimation and blurring:

./apriltag_demo -d -x 4 -b 2 270263675-464348c1-3299-400c-8907-271f70722690.pnm

gives me one detection:

detection   0: id (36x11)-37  , hamming 1, margin  186.443

but not a good edge alignment for the quad: