Conditional augmentations

[fernandocamargoai]

I've been dealing with images that varies a lot in pixel intensity. There are dark, medium and bright images. And there are images with low, medium and high contrast. To make my model deal with them, I've been using augmentations. But the problem is that I have to pick really aggressive augmentations, since the variance is too big. Then, if I use a Multiply ranging from 0.5 to 2.0, for example, the 2.0 makes my darker images look like a bright one. But the same value would destroy the information in an already bright image.

So, what I was thinking is that it would be great to condition an augmentation in some characteristics of the image. I imagine a Wrapper that takes an Augmentation and a condition and evaluate that condition to decide to apply or not that Augmentation. That way, I could have a Multiply with the appropriate range for dark images, another one for medium brightness images and another one for bright images.

What do you guys think about it?

[aleju]

If you don't need special features like random order execution, you could achieve that by calling each augmenter on its own:

import numpy as np
import imgaug.augmenters as iaa
import imgaug.parameters as iap

def augment(image):
    closeness_to_255 = np.average(image) / 255.0
    mul = iap.Normal(loc=np.log(1/closeness_to_255), scale=0.1)
    image_aug = iaa.Multiply(mul)(image=image)
    image_aug = ...  # some other method
    return image_aug

image = ...
image_aug = augment(image)

[fernandocamargoai]

@aleju, I'm currently making use of SomeOf and processing the images in batches to improve the loading performance. But your code gives me some ideas. Is there any way to make Multiply and other methods to depend on the image and still process in batches?

For the Conditional, I was thinking about forking the Sometimes and expect a lambda that takes a batch of images and returns a boolean array.

[fernandocamargoai]

Hello, @aleju. I've done the following implementation and I'm using it right now. If you want, I can make a PR to add it to the project.

https://gist.github.com/fernandocamargoti/4a8142e2489be4f9c0579df5e607aeb6

[aleju]

A PR would be great. The class looks also fine to me. The only things that I noticed were:

• in __init__ name: str = None needs to be changed to name=None as the library currently still supports python 2.7.
• func_images should probably be renamed to something like condition and expect the whole batch instance instead of just the images. That way the condition can also be based on e.g. bounding boxes ("only apply to images containing at least one 'car' object").
• The Added in 0.4.0. should be changed to Added in 0.5.0. as that will be the next official release.

But your code gives me some ideas. Is there any way to make Multiply and other methods to depend on the image and still process in batches?

You can still use the function from above. You would just have to manually remove the images that are not supposed to be augmented from the batch, before calling e.g. iaa.Multiply(...)(images=<left over images>) -- and re-add them afterwards. So it is possible, just a bit tedious. (All cases of image would also obviously have to be changed to plural images.)

[fernandocamargoai]

Alright, thanks for the feedback, @aleju. I've been very busy recently, but I'll definitely come back and make the PR soon.