How to get the masked image showing masks for object proposals?

[srbhai]

Hi, Is there a way we can get the intermediate output of the net which shows the masks on the original image representing object proposals? This would help me generate object proposals and I would like to use this on my own classifier net, trained for specific objects. Thanks.

[abbypa]

If you want to see the mask result of the net, see function evaluate_net_predictions from EndToEnd.py. It calls the net's 'predict' function, then saves the masked output in binarize_and_save_mask (ImageUtils.py)

If you want to see the original MSCOCO mask proposals (ground truths), see ExamplesGenerator.py. Here I read the given tags and create examples- positive (centered object, good size) and negative (not centered, too big/small).

[thecoldviews]

Hey! I am new to this. So given a test image does this code return a mask on all possible objects it detects in the image, just like the original implementation or is it something different? I would like to generate object proposals on an image (may include object categories not in the training set), similar to the referred paper. Would this implementation work fine to achieve this objective? Example like -

Thanks!

[abbypa]

This implementation only has the detection part- receiving a part of the image, and saying whether or not there is a centered big object in it, and if so- returns the binary mask of that object (either 0/1 or black/white). To have the full suggestions of an image, you can go over the image with a sliding window, calling the net on each part, and showing only the positive outputs. In addition, the code returns a binary mask, if you want to show the object suggestions like in the picture above (colorful & transparent) you will need some post-processing. If I remember correctly, there is a relevant function in the COCO library, but I'm not sure.

good luck!

[thecoldviews]

Thanks for the prompt response!

[thecoldviews]

Hey! I tried this code as is and it trained the model with a loss of 0.38 (both testing and training). On trying it on few the test images, all masks seem to generating as follows (which had a positive score ~0.77):

for pos-198706-591387-1-0-0-im.png

I see that your code trains the model for only one round. Is one round sufficient to achieve the accuracy marked in the paper? or did you train it for more. Could you share your evaluation results if have them?

Also above you said that the code will generate a mask only if it finds an object. The sign of the prediction tells us if it found or did not find the object right? But it generates a mask for all images even the ones with a negative score. Could you clear this out as well?

[abbypa]

The default is one round with one epoch- it is really not enough to get a good result. Try it out with more and more epochs, and stop when the result is good enough for your needs.

The mask is generated always, but should be ignored when the result indicates "no object" (meaningless)