MobilenetV3-YoloV5 gots 5x speed up on CPU compare with yolov5s

[lucasjinreal]

Hi, I did some experiement on replace backbone with Mobilenetv3, here is the model comparasion:

CPU speed:

while original yolov5s:

Tested on forward onnx.

mAP drop a little bit but aceptable due to 5x speed up on CPU inference time.

Since yolov5 target on mobile devices, does it consider using a more lightweight backbone? Still v5s is the best for now, but maybe not the lightest one.

[bharatsubedi]

@jinfagang could you please share your repository?

[lucasjinreal]

@bharatsubedi Oh, my repo is a little bit different like yolov5 does contruct model and a little mess.

[lucasjinreal]

Just tested mobilenetv3 vs yolov5s speed on TensorRT:

3ms VS 10ms

with 1280 input and GTX1080ti.

That's really fast, and also CPU speed 5x faster than yolov5s

[szad670401]

How much the mAP drop?

[lucasjinreal]

on my dataset 95 -> 90, it's still very high, so acceptable.

[glenn-jocher]

Mobilenet is in a different class of detectors, which reduce accuracy in favor of speed and size. With YOLOv5 we want useable accuracy on edge devices, something typically lacking in previous architectures like yolov3-tiny, ssdlite, mobilenet, etc.

Mobilenet v3 (https://arxiv.org/pdf/1905.02244.pdf) produces 22.0mAP@0.5:0.95 for COCO vs 37.0 for YOLOv5s currently.

[lucasjinreal]

@glenn-jocher In some tiny number classes dataset, mobilenetv3 seems OK, this is currently I got on my custom dataset. (Input resolution cares with small model)

[Alex-afka]

在我的数据集95-> 90上，它仍然很高，可以接受。

i want to study your repository? can you share it？

[ledinhtri97]

great work! @jinfagang. Could you share repository!

[lucasjinreal]

Hi, I got a better result using Ghostnet as backbone:

It's almost same with v5s, but with double speed and half model size.

model size: 29M -> 10M, CPU speed is 3x faster.

[Aktcob]

Mobilenet is better for CPU device. And for specific custom dataset, mobilenet is good choice. However, the map of COCO is much lower than yolov5s.

[bhaktatejas922]

Could you please share the (even if messy haha) repo @jinfagang ? Would appreciate it a lot. Thanks

[glenn-jocher]

I also experimented with ghostnet bottlenecks in place of standard or csp bottlenecks, but was not able to create performance increases, so we ultimately did not use them in YOLOv5. Ghostnet ops will reduce model size (and FLOPS), but will also add extra ops and layers, which in our experiments resulted in slightly slower inference and lower mAP.

[glenn-jocher]

Ghost operations are here BTW if anyone is interested in experimenting. You can use them by simply swapping a GhostBottleneck module for any of the BottleneckCSP modules in the model yaml file.

https://github.com/ultralytics/yolov5/blob/35fe98543c22e1b60f9b4f3d04c726b5aac166ef/models/experimental.py#L65-L90

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[bhaktatejas922]

Hate to ask such a basic question, but @glenn-jocher would we only need to switch out BottleneckCSP? Or do we need to switch to GhostConv as well?

[glenn-jocher]

@bhaktatejas922 I don't know what you're asking. You can use GhostConv() in place of any normal Conv().

[deeplearningers]

@jinfagang could you share it ,ths.

[bhaktatejas922]

@bhaktatejas922 I don't know what you're asking. You can use GhostConv() in place of any normal Conv().

Ah just wasn't sure if using GhostBottleneck required us to use GhostConv as well. Sorry for the confusion

[charlesyann]

Ghost operations are here BTW if anyone is interested in experimenting. You can use them by simply swapping a GhostBottleneck module for any of the BottleneckCSP modules in the model yaml file.

https://github.com/ultralytics/yolov5/blob/35fe98543c22e1b60f9b4f3d04c726b5aac166ef/models/experimental.py#L65-L90

@glenn-jocher precision drops about 5% for my dataset by using GhostBottleneck, question is why you choose kernel size 5 for self.cv2 inGhostConv, seems different with the original setting?

[glenn-jocher]

@charlesyann precision is meaningless, as it varies by confidence threshold. mAP is absolute.

[anas-899]

@glenn-jocher, I used the GhostBottleneck instead of BottleneckCSP inside "models\hub\yolov5-panet.yaml". I am getting mismatch in dimensions for the residual as shown in the image below:

for which file/model did you replace the BottleneckCSP with GhostBottleneck and worked for you? Many Thanks

[glenn-jocher]

@anas-899 you should be able to replace any C3 or BottleneckCSP layer in a model.yaml with a GhostBottleneck module. Make sure you are using the latest code, and if you run into issues I would suggest you post a reproducible example of what you did. https://docs.ultralytics.com/help/minimum_reproducible_example/

[anas-899]

Thanks @glenn-jocher for your quick response, I tried all the models (yolov5x.yaml, yolov5s.yaml, yolov5m.yaml, yolov5l.yaml, yolov3-spp.yaml, yolov3-tiny.yaml, yolov5-fpn.yaml, yolov5-panet.yaml) the only one which worked well with GhostBottleneck is yolov5-fpn.yaml (without any modification except changing BottleneckCSP with GhostBottleneck) rest of the yamls need modification in the head dimension (or by adding upsampling layer) as well. I get what I was looking for from yolov5-fpn.yaml Thanks