search

matterport / Mask_RCNN

Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow
Other
24.57k stars 11.69k forks source link

Multi-label vs Multi-class #2257

Open malgo1311 opened 4 years ago

malgo1311 commented 4 years ago

Hi,

I know that multi-class segmentation is possible with this repository. My question is regarding multi-label, where same area can be tagged with 2 labels. I am breaking down the code and in the process of modifying it for this use case, but I wanted to know if at all this will work in the end? Technically it should, but what are the challenges while inferencing or training that might not be intuitive? @waleedka - It would be great to hear what you think.

Thanks, Aishwarya

konstantin-frolov commented 4 years ago

The model detects areas in a large image, and then tries to classify and segment objects on a cropped image. In your case, I think there will be a conflict during classification. If you want to try to do this, check the same mask for the image with N classes. I have a similar problem, but I need to add images with empty masks to the training.

malgo1311 commented 4 years ago

Hi @konstantin-frolov, thank you for your reply.

I understand you are trying to feed hard negative examples, am I right? That would be interesting.

As you mentioned about the conflict during classification, I am trying exactly what you said. During classification planning to do a sigmoid instead of a softmax with all N classes. Apart from this, do you think there would be any other conflict?

konstantin-frolov commented 4 years ago

During classification planning to do a sigmoid instead of a softmax with all N classes.

This is possible with sigmoid activation, but check the 'mrcnn_class_loss_graph' in model.py to work with the new activation.

I understand you are trying to feed hard negative examples, am I right?

Yeah, I need examples of hard negatives with an empty bbox and segmentation to reduce false positives.

malgo1311 commented 4 years ago

This is possible with sigmoid activation, but check the 'mrcnn_class_loss_graph' in model.py to work with the new activation.

Thank you for the direction!