Get the pic of masks

[DeveloperFuloo]

Hi,Eruvae Actually i can obtain the massage that the masks of each object by subscribing the "/yolact_ros/detections",but i just want to get the pic of masks.Just like a pic that the foreground is white(or other colors) and the background is black.How can i do that?Thanks very much!

[crNewton]

Hi, I needed the same as you want, and there are 2 logical ways to do this:

1. publish the mask-only-image
2. don't modify yolact_ros but use /yolact_ros/detections/mask in another node.

I did method1: Modified prep_display(...) so it will return the mask-only-image, you can find it here: https://gist.github.com/crNewton/1e5ecc615b6afedeb189e3aa8d845971 (imo method 2 is better, but I couldn't get it to work for now).

@Eruvae I want to implement method 2 to use the uint8[] mask to get an contour around it in openCV, but I'm a bit lost in the masking and conversions. Do you know how we can convert the published uint8[] mask to opencv contour?

[Eruvae]

@crNewton The masks are stored as bit arrays packed with numpy's packbits function. To revert the process, you can use the unpackbits function. You should be able to use it like this:

unpacked_mask = np.unpackbits(mask_msg.mask, count=mask_msg.height*mask_msg.width)
unpacked_mask = unpacked_mask.reshape((mask_msg.height, mask_msg.width))

Note that this mask was cropped with the bounding box, so if you want it in the original image coordinates, you'd have to apply the offset specified by the bounding box (x1, y1):

image_array = np.zeros((image_height, image_width), dtype=np.uint8)
image_array[box.y1:box.y2,box.x1:box.x2] = unpacked_mask

Otherwise, you should be able to use that numpy array directly with openCV's findContours function.

@DeveloperFuloo This package currently only publishes the detections as custom ROS message, and a detection image, which is the RGB image overlaid with masks. If you want an image with just the masks, you can try modifying the prep_display function as mentioned by crNewton.