aleju / imgaug

Image augmentation for machine learning experiments.
http://imgaug.readthedocs.io
MIT License
14.43k stars 2.44k forks source link

Problem of overlapping augmented segment maps #750

Open nepersica opened 3 years ago

nepersica commented 3 years ago

Hi, I want to segment instances with imgaug but I have some problem.

With under my code, I succeed to merge several segment maps into image. result

But I can't separate merged segment maps which is augmented into origin segmentation maps' counts because of overlapping.

So I have IndexError like this. IndexError: boolean index did not match indexed array along dimension 0; dimension is 18 but corresponding boolean dimension is 19.

< augmentation.py >

relative_boxes = utils.convert_To_iaBbox(image.shape, labels['labels'], boxes)
relative_segmap = utils.convert_To_iaSegment(image.shape, masks)

seq_det = self.seq.to_deterministic()
image_aug = seq_det.augment_images([image])[0]
bbs_aug = seq_det.augment_bounding_boxes([relative_boxes])[0]
bbs_aug = bbs_aug.remove_out_of_image().clip_out_of_image()
seg_aug = seq_det.augment_segmentation_maps([relative_segmap])[0]

new_masks = utils.separate_mergedMask(image.shape, seg_aug)

boxes = utils.iaBox_To_nparray(bbs_aug)
image, masks, boxes, labels = utils.resize(image_aug, new_masks, boxes, labels)

< utils.py >

def separate_mergedMask(shape, seg_aug):  # Separate merged mask
    segmap = seg_aug.get_arr()
    index = segmap.max()
    masks = []
    for i in range(index):
        mask = np.zeros((shape[0], shape[1]), dtype=np.uint8)
        one_index = np.where(segmap == i + 1)
        mask[one_index[0], one_index[1]] = 1
        masks += [mask]

    return np.array(masks)

def convert_To_iaSegment(shape, masks):  # Merge each masks
    segmap = np.zeros((shape[0], shape[1], 1), dtype=np.uint8)  # height, width
    for i in range(len(masks)):
        mask = masks[i, :, :]
        one_index = np.where(mask == 1)
        segmap[one_index[0], one_index[1], 0] = i + 1
    segmap = SegmentationMapsOnImage(segmap, shape=shape)
    return segmap

def convert_To_absoluteBbox(shape, boxes):
    image_height = shape[0]
    image_width = shape[1]
    boxes[:, [0, 2]] /= image_width  # x coordinates
    boxes[:, [1, 3]] /= image_height  # y coordinates
    return np.array(boxes)

Is there any great idea to solve overlapping instances problem?

Thank you for reading my issue.