Polygons that have interiors

[ronif]

Hey,

First, thanks for a great module and nice features.

From my understanding imgaug doesn't currently work with polygons which have interiors. Is there some deep underlying cause or is it just in the backlog? I might want to start working on a PR for this, so would appreciate any comments.

[aleju]

Yes, interior polygons are not supported. It wasn't added because it is probably only needed by a minority of all polygon-users (which again is a minority of all users of the library) and because it would likely be a lot of work to implement. So the cost-benefit ratio isn't very good. There is also right now the workaround of using labels, e.g. "not_person" for the interior of a "person" polygon.

To implement interior polygons one would have to:

• Check all Polygon methods for whether they are affected by this and change them accordingly. This would at least affect the drawing routines, but probably most other methods too.
• Check if the methods in PolygonsOnImage would have to be changed. My guess would be no.
• Currently there is a polygon repair mechanism running after augmentation, which tries to "fix" polygons that have become invalid. That method would have to be updated. This is non-trivial as repairing invalid polygons is harder than it seems at first glance. Especially to get it fast enough that it is actually testable with many random example polygons.
• The various augmentation routines would have to be updated. This would mostly be a recursive application to the interior polygons. Still quite some work.
• All tests for the changed methods would have to be updated to verify both polygons with and without interiors. That is probably among the worst tasks of the whole job.
• The documentation would have to be updated.
• Open question: Should an interior polygon be allowed to also have an interior polygon?

I would expect this task to take a few days of work. Especially writing the tests and the automated polygon repair will likely cost a lot of time. Unfortunately I also don't see much of a way to implement a smaller minimum viable product (aside from adding a lot of assert statements in various methods to block off polygons with interior polygons, which would be rather ugly and confusing to users).

[ronif]

Thanks for the thoughtful reply.

Regarding the workaround with the labels: in my case I have polygons with interiors, which contain other polygons with interiors, so you have to be very careful with the their order to get a correct segmentation map when rasterizing.

From skimming the code it looks like the functionality can be implemented in and around _augment_polygons_as_keypoints, since only geometric augmenters work on polygons and they work by converting them to keypoints. But yeah, looks like many tests should be added.

I'll take a deeper look to see if I missed something. In any case, would you accept PR (with tests etc.) if/when submitted?

[aleju]

Sure, a PR would be welcome.

From skimming the code it looks like the functionality can be implemented in and around _augment_polygons_as_keypoints, since only geometric augmenters work on polygons and they work by converting them to keypoints.

I just looked it up and it seems like only two augmenters actually use that method nowadays (AffineCV2 -- which is deprecated -- and Lambda -- which is also fairly old). The other augmenters have custom routines to handle polygons. Though they seem to often call _apply_to_polygons_as_keypoints. In the other cases they will usually still in some way convert to keypoints.

Augmenters affected by this would be in blend.py (alpha-blending of polygons), debug.py (drawing of polygons, should be automatically done once the drawing routines are adapted to interior polygons), flip.py (horizontal/vertical flipping of polygons), geometric.py (e.g. affine transformations), meta.py (meta augmenters, like Sequential, probably nothing has to be changed here), pooling.py (probably no lines have to be changed here), size.py (resizing, padding, cropping). It might be that very few or no lines have to be changed for the augmenters, so long as the methods in Polygon are changed in the right way. Not sure about that.

[OliverColeman]

I implemented a different workaround for this, which could perhaps more easily handle arbitrary polygon constructions? https://github.com/aleju/imgaug/issues/695#issuecomment-656420930