Function interpolateCornersCharuco misplace corners

heidydallard commented 2 years ago

System information (version)

OpenCV => 4.6.0
Operating System / Platform => Ubuntu 20.04
Compiler => Python

Detailed description

With the version 4.6.0, the function interpolateCornersCharuco misplace some corners. In my case, the corners that are misplaced are the right most ones that are placed at the center of the board. The previous steps of the detection with the drawDetectedMarkers is working correctly. I have tested with the version 4.5.5 and it works in this version. result

Steps to reproduce

import copy
import matplotlib.pyplot as plt
import cv2

if __name__ == "__main__":
    dictionary = cv2.aruco.getPredefinedDictionary(7)
    charucoboard = cv2.aruco.CharucoBoard_create(29, 18, 0.01, 0.007, dictionary)
    detectorsparam = cv2.aruco.DetectorParameters_create()

    img = cv2.imread("image.png")

    arucocorners, arucoids, rejected = cv2.aruco.detectMarkers(img, charucoboard.dictionary)

    if len(arucocorners) > 0:
        imgmarkers = copy.deepcopy(img)
        cv2.aruco.drawDetectedMarkers(imgmarkers, arucocorners, arucoids)
        plt.imshow(imgmarkers)
        plt.show()

    retvalue, charucocorners, charucoids = cv2.aruco.interpolateCornersCharuco(arucocorners, arucoids,
                                                                               img, charucoboard)

    if len(charucocorners) > 0:
        imgcorners = copy.deepcopy(img)
        cv2.aruco.drawDetectedCornersCharuco(imgcorners, charucocorners, charucoids)
        plt.imshow(imgcorners)
        plt.show()

Here is the image used to do the test.

Issue submission checklist

[x] I report the issue, it's not a question
[x] I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
[x] I updated to the latest OpenCV version and the issue is still there
[x] There is reproducer code and related data files: videos, images, onnx, etc

gmedan commented 2 years ago

I've experienced a similar issue. There's been a breaking change that causes charuco board to be drawn differently in opencv 4.6.0 and consequently the object points are all wrong when you analyze a board that was generated using pre-4.6.0 code (off by one square). This is the result of the call to the following code in 4.6.0:

dictionary = cv2.aruco.getPredefinedDictionary(7)
sz_wh = (29, 18)
charucoboard = cv2.aruco.CharucoBoard_create(sz_wh[0], sz_wh[1], 0.01, 0.007, dictionary)
img_board = charucoboard.draw((sz_wh[0]*40,sz_wh[1]*40), marginSize=100)
plt.imshow(img_board, cmap='gray')

It differs from your printed board in that the first square (top left corner) is black and not white with marker. This is the same code in 4.5.5 (identical to your physical board):

I think I traced the breaking change to this commit by @AleksandrPanov, please take a look

AleksandrPanov commented 1 year ago

@heidydallard, @gmedan in the new version of OpenCV, Charuco boards are created in the same way as chess boards. Patterns with an even number of rows are now different (https://github.com/opencv/opencv_contrib/issues/3291).

PR https://github.com/opencv/opencv_contrib/pull/3305 added legacy flag to support older pattern versions. This PR fixed this problem.

jmirabel commented 1 year ago

Thank you @AleksandrPanov for your support.

opencv / opencv_contrib