Bounding box vs instance segmentation

[OAT7963]

❔Question

Hi, should I use bounding boxes or instance segmentation to label my training data if I want to use the model to detect and classify vehicles in case of traffic congestion. Which would yield better result?

Below is an traffic congestion image example.

Thanks

[github-actions[bot]]

Hello @OAT7963, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[glenn-jocher]

@OAT7963 the best way to find out is to try both and then work from empirical results.

[whoafridi]

I would say both are (bounding boxes or instance segmentation) different set of problems where for boxes you use rectangle for data labels & segmentation use polygon. In addition both have different format of annotations also. So, the best way to find out is to try both and then work from empirical results. Possibly segmentation looks great for traffic congestion problem instead of bbox .

[glenn-jocher]

We did a traffic demo earlier here: https://www.youtube.com/watch?v=iUpZagGyhlM&ab_channel=UltralyticsLLC

Though of course the angle of the camera will impact results, among other things.

[MikeHatchi]

Hi Everyone,

Then, I've probably missed something. Yolov5 asks for training: class x_center y_center width height, not any color_class (or color_category) data thus @OAT7963 needs to add this input parameter from scratch within the model, doesn't he?

[glenn-jocher]

@MikeHatchi yes for @OAT7963 to train segmentation tasks significant feature additions would be required to the YOLOv5 repository (loss function, dataloader, testing methodology), in addition to model architectural changes (a head suitable for segmentation outputs).

The COCO dataset itself already supports segmentation training as it provides segmentation labels.

[MikeHatchi]

@glenn-jocher Thx!!

FYI, I've made this little pull request: https://github.com/vrona/yolov5/pull/1, as if not it raised "Key Error" when lauching training.

[glenn-jocher]

@MikeHatchi ah, yes this was a recent breaking change in master, it’s mentioned in the 3.1 release notes.

Note that you submitted a PR to a fork though rather than here.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.