search

aleju / imgaug

Image augmentation for machine learning experiments.
http://imgaug.readthedocs.io
MIT License
14.42k stars 2.45k forks source link

Requiring that keypoints stay within the image bounds #78

Open isarandi opened 6 years ago

isarandi commented 6 years ago

A feature idea. Not sure if it fits within the scope of this library or should be rather handled by the user, but this comes up quite often when using augmentation: We want to warp the image but not so much that important bits (keypoints) fall out of frame.

One solution is to carefully craft the ranges of translation/scaling/rotation for the particular dataset and hope that most keypoints will remain within the image frame after augmentation.

Another approach, that I take now, is to sample new augmentations with aug.to_deterministic() in a loop until I get one where all keypoints remain within frame. This can of course slow things down significantly depending on the configuration. It may also happen that there doesn't even exist such a transform, so the number of trials should be limited.

This may be useful to have within imgaug, for example as a boolean argument to the augmenter functions that take keypoints as parameters (augment_keypoints, augment_batches). For example with a name like keep_keypoints_visible or force_keypoints_within_image_bounds or something better.

aleju commented 6 years ago

Setting reasonable ranges for the parameters and dropping bad augmentations is probably the best one can do with the library right now. Changing this to a parameter like keep_keypoints_visible would likely be very difficult to get into the library, because currently images and keypoints are augmented in completely separate steps and are only "synchronized" by the determinism, while such a parameter would require some kind of one-step handling. It would also break the current philosophy that parameters should only be sampled conditionally dependent on the random state and image shape.