zivid / zivid-python

Official Python package for Zivid 3D cameras
BSD 3-Clause "New" or "Revised" License
40 stars 14 forks source link

Add marker detection and hand-eye API wrappers #276

Closed johningve closed 2 months ago

johningve commented 2 months ago

This commit adds API wrappers for the SDK 2.13 fiducial marker detection and hand-eye calibration.

Added to the zivid.calibration module are the classes MarkerShape, MarkerDictionary, and DetectionResultFiducialMarkers, as well as the function detect_markers. The DetectionResultFiducialMarkers class can be used to construct a HandEyeInput, just like DetectionResult.

The hand-eye test data was updated with some newer ZDFs that were downsampled to save on Git LFS bandwidth.

torbsorb commented 2 months ago

Please consider changing the format of corners_in_pixel_coordinates in order to simplify the following (which is currently required):

rgba = camera_frame.point_cloud().copy_data("rgba")
rgb = rgba[:, :, :3].copy().astype(np.uint8)
if marker_list is not None:
    detected_markers = detection_result.detected_markers()
    detected_corners = [
        np.array(marker.corners_in_pixel_coordinates).reshape((4, 1, 2))
        for marker in detected_markers
    ]
    marker_ids = np.array([marker.id for marker in detected_markers])
    rgb = cv2.aruco.drawDetectedMarkers(rgb, detected_corners, marker_ids, [0, 255, 0])
johningve commented 2 months ago

@eskaur @Frontier789 could you advise on this?

johningve commented 2 months ago

Do we have the same issue in C++ then? Or is this just a problem in python?

eskaur commented 2 months ago

Do we have the same issue in C++ then? Or is this just a problem in python?

In C++ we return our own special types (std::array<Zivid::PointXY,4>), so the user has to do some conversion before passing it to OpenCV anyway. Therefore I don't consider this an issue in C++. In python it becomes more tempting to be super compatible with 3rd party APIs since we return universal stuff like lists and numpy arrays.

eskaur commented 2 months ago

Working on making the corner getters return numpy arrays.