Avoid 1-hot encoding in SegmentationMapOnImage

[martinruenz]

SegmentationMapOnImage converts masks to a 1-hot representation (see https://github.com/aleju/imgaug/blob/69ac72ef4f2b9d5de62c7813dcf3427ca4a604b5/imgaug/imgaug.py#L4825).

However, this has two drawbacks:

• It is required to know the (at least upper bound) number of classes
• The representation is wasteful and, especially when N is large

I guess the conversion is performed to make drawing easier. I would suggest to stick to the original representation and to only convert when the user is drawing. (Commonly drawing samples is expensive anyway and avoided in training loops)

[aleju]

Yes, this is currently inefficient. Can't remember why exactly I implemented it this way - it must have made something easier. It is already added to the TODO list. :smile:

[ywpkwon]

In addition, even when I only need 2D transform (such as CropAndPad) for segmentation label (which is nothing but just cutting/padding/resizing/etc. a 2D integer gray image, and it may never relate to how large number the gray image contains), the current implementation takes longer when nb_classses is large. Hope this part gets improved. Thanks for this awesome library.

seq = iaa.Sequential([iaa.Fliplr(0.5),
                          sometimes(iaa.Affine(
                              scale={"x": (0.99, 1.01), "y": (0.99, 1.01)}, # scale images to 80-120% of their size, individually per axis
                              translate_percent={"x": (-0.05, 0.05), "y": (-0.05, 0.05)}, # translate by -20 to +20 percent (per axis)
                              rotate=(-3, 3), # rotate by -45 to +45 degrees
                              shear=(-2, 2), # shear by -16 to +16 degrees
                              order=[0, 1], # use nearest neighbour or bilinear interpolation (fast)
                              cval=(0, 0), # if mode is constant, use a cval between 0 and 255
                              mode=["constant"] # use any of scikit-image's warping modes (see 2nd image from the top for examples)
                          )),
                          sometimes(iaa.OneOf([
                              iaa.GaussianBlur((0, 3.0)),
                              iaa.AverageBlur(k=(2, 7)),
                          ])),
                          sometimes(iaa.CropAndPad(
                              percent=(-0.25, 0.00),
                              pad_mode=["constant"],
                              pad_cval=(0, 0)
                          )),
                         ])
...
segmap = ia.SegmentationMapOnImage(label, shape=image.shape, nb_classes=NUM_CLASSES)  # nb_classes should'nt matter
seq_det = seq.to_deterministic()
image_aug = seq_det.augment_image(image)
segmap_aug = seq_det.augment_segmentation_maps([segmap])[0]

[georgezakinih]

Hi @aleju ,

Thanks for the great work! I wonder if this issue is addressed yet? I am augmenting masks for instance segmentation where images can have few hundreds instances and it is currently taking super long time to augment the masks.

George

[aleju]

Not addressed yet. I can see that instance segmentation is infeasible with the current method. Version 0.2.9 will be released soon and the version after that will likely fix this problem.