3d (non-planar) Aruco board - Sometimes "mirrored" pose is detected

[gayoso]

OpenCV: 4.1.2

Im doing some pose estimating on a 3d object with aruco markers, defining a non planar aruco board from the markers. It is working great, but i noticed that when only certain two markers are visible, the "mirrored" pose is detected instead of the correct one. This only happens on a narrow rotation range, when both markers appear more frontal to the camera. I made a simplified model with only those two markers.

This is how it should estimate:

And this is how it estimates for that range:

I imagine this is because those two markers in particualr have some symmetry. Is there anything that can be done to avoid this while keeping the markers as they are? (without adding markers or rotating them to break the symmetry)

EDIT: I should add that the pose does not flicker, it is stable and mirrored in that range.

Would it be possible to define the axes of the aruco board so that the "mirrored" pose is never considered, but only poses than can be achieved with rotations on those original axes?

Thanks!

[alalek]

Please provide complete minimal reproducer including original non-modified input data (as .png images)

---

OpenCV: 4.1.2

You need to upgrade.

[gayoso]

opencv_aruco_mirrorpose_minimal.zip https://pastebin.com/Q4bUyMPh

Hi, I upgraded to OpenCV 4.5.5 and I am observing the same behaviour. The provided .zip file or pastebin contains the code for a minimal example in which this happens:

https://user-images.githubusercontent.com/7811510/155454565-f945d9e9-6a6f-4fc6-889e-00de9d60ca86.mp4

I can provide the .stl to print the object in the video if it would be of use. Let me know if there is anything else I should try.

[AleksandrPanov]

opencv_aruco_mirrorpose_minimal.zip https://pastebin.com/Q4bUyMPh

Hi, I upgraded to OpenCV 4.5.5 and I am observing the same behaviour. The provided .zip file or pastebin contains the code for a minimal example in which this happens:

aruco_mirrorpose.mp4 I can provide the .stl to print the object in the video if it would be of use. Let me know if there is anything else I should try.

@gayoso, сould you attach the non-modified input data (PNG or mp4)?

[AleksandrPanov]

This issue may be related to issue https://github.com/opencv/opencv/issues/8813.

[gayoso]

It does look similar to that issue. I'll keep an eye on it.

Here is the code to run from a video as input: https://pastebin.com/WAWzySKn And the video: https://user-images.githubusercontent.com/7811510/155592975-2caff765-2969-4575-b818-0f48ebcd7173.mp4

[powellsky]

This issue may be related to issue opencv/opencv#8813.

I am assuming that this might be the case. I was testing aruco boards with 2 + markers. When I used 2 I had very unstable results, just like with using only one marker. After defining boards with more markers the problem was resolved. This might be the case of ambiguity which is a common problem for solvePnP algorithm (it is not a bug!). All corners of your markers are passed to solvePnP function behind the scenes, and it might be the case those 8 points in space could be not enough to solve ambiguity. I know that you do not want to add more markers, but this is the only thing that comes to my head.

[crackwitz]

documentation doesn't specify the order of corners properly. it merely says "CCW", which I think is specifically wrong, because aruco corner points are usually specified top left, top right, bottom right, bottom left, with X right, Y far/up (Z up/near).

also I would question those magic constants in your reproducer code. best to use transformations to arrive at those values given nominal parameters (size of marker, nominal translation values as used in the CAD model for that 3D-printed fixture).

this has helped me: https://docs.opencv.org/4.x/db/da9/tutorial_aruco_board_detection.html