ethz-asl / rovio

Other
1.12k stars 506 forks source link

Add an optional image mask for feature detections #206

Open helenol opened 5 years ago

helenol commented 5 years ago

In case you forget to mount your camera so that it doesn't look at your propellers or your frame. ;) Simply prunes new feature detections before selecting best features to track based on whether their coordinates fall into the mask or not.

Before: image

After: image

Mask applied: (black = filter, white = don't filter) gonzen_mask_big

Controlled by an optional ros param called "image_mask_path". If set to a path (here a png image), will automatically filter. Otherwise won't do anything.

mhkabir commented 5 years ago

This should also be passed through to the FAST extractor call so that it doesn't detect points on masked regions in the first place.

helenol commented 5 years ago

@mhkabir this is pruning the output of the fast feature detector to discard all features that are in the masked regions. Masking the image before putting it through the feature detector would result in features on the mask boundaries, which we don't want.

mhkabir commented 5 years ago

I wasn't suggesting masking the image manually before passing it. What I meant was, you should pass the mask Mat through to the FAST detector call (https://github.com/ethz-asl/rovio/blob/master/include/rovio/ImagePyramid.hpp#L142). There is already an API: image

This does not cause detections on the edges.

helenol commented 5 years ago

@mhkabir Ah cool thanks a lot! Ok will see if it's faster to just switch to that.

mfehr commented 5 years ago

We added masking to the maplab_rovio version, based on this PR. However we refactored and extended it a bit, when we discovered some issues with this masking, as tracked features were still entering masked areas and then stayed for several frames (+ we added the keypoint masking suggested by @mhkabir ). Any feedback would be welcome :)