Inference performance with different backbone

matterport / Mask_RCNN

Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow

Other

24.66k stars 11.7k forks source link

Inference performance with different backbone #1420

Open fbottarel opened 5 years ago

fbottarel commented 5 years ago

Hey folks,

I trained and tested a bunch of models with this implementation. Everything works smooth, I'm just wondering if I'm doing something wrong since the results I'm getting are not in line with the original paper. In particular, I'm talking about inference time:

I'm getting about 5 FPS when performing batch inference, regardless of batch size (nVidia P100)
Performance is even lower (~2 FPS) when performing inference from a live camera stream.

I should also mention that I changed the architecture backbone to resnet50.

Am I missing something obvious here? Did anyone experience the same issues?

bjornamr commented 5 years ago

@fbottarel

Hi, have you tried using MobileNet for inference? Maybe this would speed up things?

fbottarel commented 5 years ago

Thanks @bjornamr for addressing my call. Could you point me in the right direction in how to set that up? I believe MobileNet is not already included in this implementation (I am not too familiar in messing with backbones yet)

bjornamr commented 5 years ago

@fbottarel I have not looked into the code too much myself. I might do it later this year, but I dont know when. I might need mask-rcnn with higher performance later too.

I think a good starting point is looking at the code here: https://github.com/matterport/Mask_RCNN/blob/1ad9feaae3d87b52495413e6c8ea0e92f0e5bc34/mrcnn/model.py

Please post if you have any progress or questions. Maybe me or anyone else can help.

bjornamr commented 5 years ago

@fbottarel I forgot to ask, do you need the masks or do you only need the bounding boxes?

fbottarel commented 5 years ago

Masks are important for my application! I have been looking into the subject, and I'm afraid that MobileNet, while definitely faster, would not be able to extract features that can give good quality segmentation. For now this is moved in backlog, at least until I have studied the state of the art or something else pops up...

Should I close the issue?

bjornamr commented 5 years ago

I would also love to see that here, but I do not have the time right now. I might look into it later myself if we need it at work. I think we do later this year.

You might also look into fbNet here: https://github.com/facebookresearch/maskrcnn-benchmark/blob/master/MODEL_ZOO.md

It is not TensorFlow, but Pytorch.

ApoorvaSuresh commented 5 years ago

Hi,

I am using mobilenet backbone, but getting this error while inference:

ValueError: Layer #1 (named "conv1") expects 2 weight(s), but the saved weights have 1 element(s).

Any idea why?

LIMU2 commented 5 years ago

Hi,

I am using mobilenet backbone, but getting this error while inference:

ValueError: Layer #1 (named "conv1") expects 2 weight(s), but the saved weights have 1 element(s).

Any idea why?

Hi, may I ask how to use mobilenet backbone? I looked into code and it's said:" # Supported values are: resnet50, resnet101."

Adithia99 commented 4 years ago

do you have codes how to detect video with FPS? i can detect it in realtime but dont know how to display the FPS