Dotted Line Detection and Segment Merging via Dotted Line

[ErenKarabey]

The example in this issue are from:

• Knapp S (2013) A revision of the Dulcamaroid Clade of Solanum L. (Solanaceae) PhytoKeys 22 1-428.
• https://doi.org/10.3897/phytokeys.22.4041

figure 8: and figure 10:

The new tested segmentation and labelling algorithm is given below.

def testf(image, predictions, min_area=500):
    gray = cv2.cvtColor(image, 6)

    _, binary = cv2.threshold(gray, 250, 255, cv2.THRESH_BINARY_INV)

    contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

    contours = [cc for cc in contours if cv2.contourArea(cc) > min_area]

    contours.sort(key=lambda cc: cv2.contourArea(cc), reverse=True)

    label_contour_map = contours2Label(contours, predictions)
    # colors = colorLogic(len(label_contour_map))

    kernel_size = 3
    op_iterations = 3
    morph_kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (kernel_size, kernel_size))

    fill_position = (0, 0)
    fill_color = (255, 255, 255)
    color_tolerance = (0,0,0)

    copy = np.copy(image)
    segments = []
    for i, key in enumerate(label_contour_map):
        # color = colors[i]
        binary_copy = np.copy(binary)
        for cnt in label_contour_map.get(key, []):
            cv2.drawContours(binary_copy, [cnt], 0, (0, 0, 0), -1, 8)

        segment = binary - binary_copy
        # showImage(segment)
        binary_mask = cv2.morphologyEx(segment, cv2.MORPH_DILATE, morph_kernel, None, None, op_iterations,cv2.BORDER_REFLECT101)

        blob_mask = cv2.bitwise_and(copy, copy, mask=binary_mask)

        cv2.floodFill(blob_mask, None, fill_position, fill_color, color_tolerance, color_tolerance)

        masked_image = cv2.bitwise_or(image, blob_mask)

        binary = binary - segment

        segments.append(masked_image)

    return segments

Results for Figure 8: Results for Figure 10:

The dotted lines are never taken into consideration by the algorithm. This is case for both segmentation, and classification.

For segmentation, dotted-line contour may be separate and even if its brought together by dilation, its contour area is always smaller than labelling letter contour area - which is added later in classification with no chance of error and a very good indicator of min_area). However, the resulting segments should strive to show the dotted-lines as they portray important information about the monograms.

For classification, in Figure 8, even without considering the dotted-lines classification has no error. However, there are some cases such as Figure 10, where the classification D contains a part of classification E - or as there is an intersection and the algorithm does not detect intersections, it should be classified as A. The intersection will be discussed extensively in another issue.

Both for segmentation and for classification dotted-lines are necessary and should be detected.

[ErenKarabey]

The reason Figure 10 Label D contains extra is the classification algorithm is using shortest line distance to determine what to classify each segment as. In some cases, such as Figure 10 Label D, the dotted-line section is closer to a wrong label (D) than the correct one (E).