keras-team / keras-cv

Industry-strength Computer Vision workflows with Keras
Other
998 stars 328 forks source link

Vectorize (+ randomize) Posterization. #1557

Open sebastian-sz opened 1 year ago

sebastian-sz commented 1 year ago

Short Description Posterization could also benefit from vectorization and using factor sampler. This would not change performance much, but could allow to apply different augmentations per sample (currently this is not possible).

I propose the following changes:

1. Properly vectorize this layer Currently the vectorization is done by a hacky approach: https://github.com/keras-team/keras-cv/blob/57b5c1e0cd9fb95824fcce5051767470a8e37fc7/keras_cv/layers/preprocessing/posterization.py#L106-L109

The above is misleading and VectorizedBaseImageAugmentationLayer is a much cleaner alternative.

2. Add random behaviour This layer could accept bits_factor with values in range 1-8, and apply different augmentations per image. The deterministic, same usage could still be achieved via bits_factor=(bits, bits).

This is however, backward incompatible so I propose to rename this layer to RandomPosterization to signal these changes.

Let me know what you think!

sebastian-sz commented 1 year ago

The performance is similar, maybe slightly lower (I suspect due to random behaviour).

Layer Mode N=1000 N=2000 N=5000 N=10000
OldPosterization Eager 0.014 0.016 0.074 0.144
RandomPosterization Eager 0.010 0.018 0.086 0.154
OldPosterization Graph 0.014 0.016 0.071 0.146
RandomPosterization Graph 0.015 0.019 0.079 0.150
OldPosterization XLA 0.012 0.014 0.072 0.134
RandomPosterization XLA 0.013 0.013 0.073 0.139

comparison comparison_no_old_eager

Please let me know what do you think about this approach - I can submit a PR.