Slow Processing in BadPhotoCopy Augmentation

[shaheryar1]

I ran the notebook to reproduce the noisy image. However, the badphotocopy augmentation is taking more than 3 minutes on an image with shape (2200, 1700, 3) .

[kwcckw]

Yes, right now it's slow because the hashing process is pixel by pixel and it is done with more than 1 iterations using loops, it's get faster if we are using a smaller size mask (hashing map), but sometimes we need larger mask to create certain effect. I think we can improve it by using vectorization, so that should be in the next update roadmap.

[proofconstruction]

Let's prioritize this performance increase. Augraphy is designed for producing many documents for training data, so if we have a 3 minute delay on a single pipeline and need to run thousands of pipelines, it's a huge problem

[kwcckw]

Let's prioritize this performance increase. Augraphy is designed for producing many documents for training data, so if we have a 3 minute delay on a single pipeline and need to run thousands of pipelines, it's a huge problem

Sure, then i will look into this next.

[kwcckw]

So i've tried np.vectorization or using parallel processing but looks like the result more or less in a same speed and parallel processing is slower in this case.

Example with vectorization, original:

for i, m in np.ndenumerate(pixels):
        y,x = i

        hash_value = self._hash_random(
            x + x_within_field,
            y + y_within_field,
            iteration
        )
        pixels[i] += hash_value

With vectorization:

    coordinates = list(itertools.product(range(pixels.shape[0]), range(pixels.shape[1])))
    ys,xs = map(list,zip(*coordinates))

    ys_within_field = ys + y_within_field
    xs_within_field = xs + x_within_field
    iterations =  np.full(len(ys), iteration)

    hash_random_vectorize = np.vectorize(self._hash_random)
    hash_value = hash_random_vectorize(xs_within_field, ys_within_field, iterations)

    for i, coordinate in enumerate(coordinates):
        pixels[coordinate] += hash_value[i]

Do you guys having any other suggestion on how to speed up the process especially for for loop?

[jboarman]

I've not read through the code, but perhaps it would be informative to see how Imgaug implements its cloud/fog augmentation?

Docs https://imgaug.readthedocs.io/en/latest/source/overview/imgcorruptlike.html#fog https://imgaug.readthedocs.io/en/latest/source/overview/weather.html#clouds

Source https://github.com/aleju/imgaug/blob/master/imgaug/augmenters/weather.py

[kwcckw]

Thanks, I will go through it first, the worst case is we might need to rewrite the whole algorithm since it's too slow.

[kwcckw]

The current Badphotocopy algorithm is based on this repo: https://github.com/tatarize/noisemaker

It took me around 60 seconds to run the original code with image size of 1500.

I found another library which is using Perlin noise algorithm and it uses around 3 seconds. So at this point, i guess 3 seconds per image should be fast enough right?

Some examples of Perlin's noise mask:

[proofconstruction]

A reduction from 60 seconds to 3 seconds is a massive improvement :tada: great work!

The average default pipeline run is something like 4.5 seconds on my machine, and when users want to create large datasets for training, they should use multiple processes to run several pipelines concurrently. We may have to revisit performance at a later date, but I think every augmentation is fast enough for now.

[jboarman]

That is definitely a huge improvement!

I’d say the measure of whether it’s “fast enough” comes down to how it compares to Imgaug’s implementation speed for their noise augmentation (Cloud / Fog). I have not run their augmentation personally, so I’m not sure what that baseline is. Has anyone had a chance to run theirs to see how the speed compares?

[kwcckw]

I tried out their test code here: https://github.com/aleju/imgaug/blob/master/test/augmenters/test_weather.py

    imgs_aug = iaa.Fog().augment_image(img)
    imgs_aug = iaa.Rain().augment_image(img)
    imgs_aug = iaa.Clouds().augment_image(img)
    imgs_aug = iaa.Snowflakes().augment_image(img)

In my machine, i can see the time taken is range from 1.x seconds to 2.x seconds, i guess that's the average time, but they did support batch processing, and that may speed up the time further since some libraries need to call only once. For example, with 1 image: imgs_aug = iaa.Fog().augment_image(img), with multiple images: imgs_aug = iaa.Fog().augment_images([img, img2, img3]).

Here are the observed outputs (fog, rain, cloud, snowflakes), but i don't see any augmented effect in snowflakes:

I think we can have similar "rain" effect in our repo too, something like "water-sokaed document". For reference, this is the zoom in view from their Rain agumentation:

[kwcckw]

Sorry, to make it clear, 3 seconds are the time to generate Perlin mask, so with other process, it would still take more than 3 seconds, i will edit it and remove some unnecessary sections again later.

[kwcckw]

This should be resolved now, please close the issue.