qubvel / segmentation_models

Segmentation models with pretrained backbones. Keras and TensorFlow Keras.
MIT License
4.76k stars 1.03k forks source link

Prediction mask overlay like Mask-RCNN. Is it possible? #314

Open Sphincz opened 4 years ago

Sphincz commented 4 years ago

Is it possible to overlay a predicted mask over the original image? I'm struggling to find a way around your examples in order to achieve a result very similar to this:

image

VladislavAD commented 4 years ago

You can use opencv to achieve this. First of all you need layers with your answer, if you are using python then your answer is already numpy array (batch, size, size, layers). Then you just need to paint this layers over original image. So you take layer, make mask of it (cast it to separate single channel image), multiply by color you want, stack it into three channels and add to image.

# colors is the list of predefined colors if RGB format like (255,0,0) is red
for i in range(classes):
    # answers are the list of layers of predicted labels (size, size) or (size, size, 1), we create rgb image by stacking single chanel label three times myltiplied by r, g and b color value
     answers[i] = np.stack((colors[i][0] * answers[i], colors[i][1] * answers[i],colors[i][2] * answers[i]), axis=-1).astype(np.uint8)
    # next we use addWeighted from opencv to add colors to original image
     result_img = cv2.addWeighted(result_img,1.0,answers[i],0.7,0)

Not the best solution, but it works. You can use other libs like pillow to paint over image but I prefer opencv because it works with numpy arrays. Next you can save image using cv2.imwrite or for example show it using matplotlib which works fine with numpy arrays.