[QST] skimage draw module

[monzelr]

I have searched for an alternative cv2.fillPoly function and found the skimage.draw.polygon2mask

here in cucim, I was wondering that the whole skimage.draw module is missing. is there any reason that it is not supported or is it just not implemented?

I have seen, that the performance of cv2.fillPoly is pretty good (around 10ms for FHD image) - however, GPU accelerated would it beat in factor 10 is my guess.

[grlee77]

Hi @monzelr

We have so far not tried to implement any of the skimage drawing functions in cuCIM. There is another recent NVIDIA project called CV-CUDA which is providing APIs more similar to OpenCV than scikit-image (i.e. mainly supporting 2D images with 1, 3 or 4 channels and not providing n-dimensional support). It does provide one drawing-related operator named cvcuda.osd (where OSD is an acronym for "on screen display"). This operator is based on drawing into an array of pixels via CUDA kernels. I have not done any type of benchmarking of it in terms of performance, but would be interested if you try it and have feedback.

While it is still in beta, I think the CV-CUDA wheels are not on PyPI and instead need to be downloaded from the "Assets" section on their release page (use variants with "cu12" in the wheel name if you have CUDA 12.x, "cu11" for CUDA 11.x).

CV-CUDA is still in beta and doesn't have a lot of examples in the docs, but here is a basic example of how to use this function to draw elements of various types (bounding box, line, polygon, etc.) onto an existing CuPy ndarray.

import cupy as cp
import cvcuda
import matplotlib.pyplot as plt
import numpy as np

# dark gray background
rgb_img = cp.full((400, 700, 4), 40, dtype=cp.uint8)
# fully opaque background
rgb_img[..., 3] = 255

# zero-copy conversion of the cupy.ndarray to a nvcv.Tensor (cvcuda.Tensor)
cv_tensor = cvcuda.as_tensor(rgb_img, layout="HWC")

# Configure desired on-screen display elements via cvcuda.Elements
osd_elements = cvcuda.Elements(
    elements=[
        [
            cvcuda.BndBoxI(
                box=(10, 10, 35, 35),
                thickness=2,
                borderColor=(255, 255, 0),
                fillColor=(0, 128, 255, 128),  # blue semi-transparent
            ),
            cvcuda.BndBoxI(
                box=(200, 50, 30, 25),
                thickness=1,
                borderColor=(255, 0, 0),
                fillColor=(0, 0, 0, 0),  # fully transparent fill
            ),
            cvcuda.Label(
                utf8Text="label",
                fontSize=16,
                tlPos=(450, 250),
                fontColor=(255, 255, 0),
                bgColor=(0, 128, 255, 128),
            ),
            cvcuda.Segment(
                box=(80, 20, 50, 50),
                thickness=1,
                segArray=np.array(
                    [
                        [0, 0, 0, 0, 0.2, 0.2, 0, 0, 0, 0],
                        [0, 0, 0, 0.2, 0.3, 0.3, 0.2, 0, 0, 0],
                        [0, 0, 0.2, 0.3, 0.4, 0.4, 0.3, 0.2, 0, 0],
                        [0, 0.2, 0.3, 0.4, 0.5, 0.5, 0.4, 0.3, 0.2, 0],
                        [0.2, 0.3, 0.4, 0.5, 0.5, 0.5, 0.5, 0.4, 0.3, 0.2],
                        [0.2, 0.3, 0.4, 0.5, 0.5, 0.5, 0.5, 0.4, 0.3, 0.2],
                        [0, 0.2, 0.3, 0.4, 0.5, 0.5, 0.4, 0.3, 0.2, 0],
                        [0, 0, 0.2, 0.3, 0.4, 0.4, 0.3, 0.2, 0, 0],
                        [0, 0, 0, 0.2, 0.3, 0.3, 0.2, 0, 0, 0],
                        [0, 0, 0, 0, 0.2, 0.2, 0, 0, 0, 0],
                    ]
                ),
                segThreshold=0.2,
                borderColor=(255, 255, 255),
                segColor=(0, 128, 255, 128),
            ),
            cvcuda.Point(
                centerPos=(100, 100),
                radius=4,
                color=(255, 255, 0),
            ),
            cvcuda.Line(
                pos0=(80, 85),
                pos1=(150, 75),
                thickness=3,
                color=(255, 0, 0),
            ),
            cvcuda.PolyLine(
                points=np.array(
                    [
                        [120, 120],
                        [300, 100],
                        [250, 140],
                        [350, 180],
                        [400, 220],
                    ]
                ),
                thickness=0,
                isClosed=True,
                borderColor=(255, 255, 255),
                fillColor=(0, 255, 128, 96),
            ),
            cvcuda.RotatedBox(
                centerPos=(130, 180),
                width=22,
                height=22,
                yaw=0.3,
                thickness=1,
                borderColor=(255, 255, 0),
                bgColor=(0, 128, 255, 128),
            ),
            cvcuda.Circle(
                centerPos=(540, 30),
                radius=25,
                thickness=2,
                borderColor=(255, 255, 0),
                bgColor=(0, 128, 255, 128),
            ),
            cvcuda.Arrow(
                pos0=(550, 200),
                pos1=(450, 100),
                arrowSize=12,
                thickness=3,
                color=(0, 255, 255, 128),
            ),
            cvcuda.Clock(
                clockFormat=cvcuda.ClockFormat.YYMMDD_HHMMSS,
                time=0,
                fontSize=10,
                tlPos=(100, 350),
                fontColor=(255, 255, 0),
                bgColor=(0, 128, 255, 128),
            ),
        ],
    ],
)

# draw the desired elements onto src, storing the result in cv_out
# The draw operations are done via CUDA kernels.
# Note: there is also a `cvcuda.osd_into` to write into a pre-allocated output array instead of allocating a new one
cv_out = cvcuda.osd(src=cv_tensor, elements=osd_elements)

# cv_out.cuda() exports a buffer supporting DLPack and __cuda_array_interface__
out = cp.asarray(cv_out.cuda())

# copy back to CPU to plot
plt.figure()
plt.imshow(cp.asnumpy(out))
plt.show()

which takes a dark gray image and draws elements onto it so that it ends up looking like this