On the understanding of the yolov8-seg model

[xiaocheng99]

Search before asking

• [X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hello, so far I know that YOLOV8-seg can be seen as a multitasking perceptual model, so in the head section is it equivalent to having both a detection head and a segmentation head? I am very confused here. In my own project, the dataset contains both polygon and rectangular box labeled values, and performs the detection and segmentation tasks separately.1. According to you, yolov8-seg is in fact a multitasking perceptual task, and I'm in the part of the model evaluation where I'm modifying the evaluation script to separate the metrics for detection (rectangle labeling) and segmentation (polygon labeling) and applying appropriate metrics calculations for each prediction. How should I accomplish this if I use yolov8-seg directly to implement the functionality I need for the relevant comparison tests? For the target detection part, extract the rectangular box labels separately and use some other target detection models to complete the comparison experiment; extract the polygon labels in the labeled file separately and use other segmentation models to complete the comparison experiment, is that how I should do it? 2. Or should I imitate yolop, and add a detection header on top of yolov8-seg to realize the functions I need? or should I copy yolop and add a detection header to yolov8-seg to realize the function I need? I am very eager to get your answer, I can't figure it out here.

Additional

No response

[xiaocheng99]

A very earnest request for an answer, does the head part of the yolov8-seg model contain both the detection header and the segmentation header? Or do I need to add another detection header into it if I want to perform detection + segmentation tasks.

[hrnbot]

Yes, the YOLOv8-seg model is the multi tasking perceptual model it performs detection and instance segmentation both. in yolov8-seg head is designed to handle both tasks, producing bbox for detection as well as pixel wise masking for the instance segmentation.

For your Project you have rectangle and polygon both , you can use yolov8 with only polygon to do segmentation.

Below is small training you can see the box and mask both have P,R and mAP. You can add other matrix over there. In yolo training you will get matrix for detection as well as segmentation. Class Images Instances Box(P R mAP50 mAP50-95) Mask(P R mAP50 mAP50-95): all 9 38 0.548 0.488 0.494 0.219 0.559 0.589 0.505 0.191 c 9 27 0.699 0.63 0.652 0.395 0.592 0.63 0.579 0.246 b 9 8 0.633 0.5 0.586 0.218 0.599 0.5 0.591 0.241 c 9 3 0.313 0.333 0.244 0.0443 0.485 0.637 0.344 0.0874

[xiaocheng99]

Yes, the YOLOv8-seg model is the multi tasking perceptual model it performs detection and instance segmentation both. in yolov8-seg head is designed to handle both tasks, producing bbox for detection as well as pixel wise masking for the instance segmentation.

For your Project you have rectangle and polygon both , you can use yolov8 with only polygon to do segmentation.

Below is small training you can see the box and mask both have P,R and mAP. You can add other matrix over there. In yolo training you will get matrix for detection as well as segmentation. Class Images Instances Box(P R mAP50 mAP50-95) Mask(P R mAP50 mAP50-95): all 9 38 0.548 0.488 0.494 0.219 0.559 0.589 0.505 0.191 c 9 27 0.699 0.63 0.652 0.395 0.592 0.63 0.579 0.246 b 9 8 0.633 0.5 0.586 0.218 0.599 0.5 0.591 0.241 c 9 3 0.313 0.333 0.244 0.0443 0.485 0.637 0.344 0.0874

Hi, so I only use yolov8-seg to accomplish detection + segmentation at the same time, is there any difference between this and yolop? Do I need to add a detection header to the head of the yolov-seg to realize detection + segmentation? Or does the model already have a detection header + segmentation header in the head part?

[glenn-jocher]

@xiaocheng99 yes, YOLOv8-seg is designed to perform both detection and segmentation tasks simultaneously. The model's head contains components for both bounding box detection and pixel-wise segmentation, so there's no need to add an additional detection header. It's an integrated solution that should meet your requirements for handling both rectangle and polygon annotations in your project. The performance metrics for both detection and segmentation are provided during training and validation, allowing you to evaluate the model's effectiveness for each task. 🚀

[joshua-atolagbe]

Hello @glenn-jocher . Please how do I modify the architecture of yolov8-seg, especially the detection branch? The existing architecture is not performing well on my custom dataset. Also I want to train the model using v8Segmentation loss.

[glenn-jocher]

@joshua-atolagbe hello! To modify the architecture of YOLOv8-seg, you'll need to adjust the model configuration files where the layers and hyperparameters are defined. For the detection branch, focus on the layers responsible for bounding box predictions. To train with the v8Segmentation loss, ensure that your dataset is properly annotated for segmentation and that the loss function is specified in the training configuration. For detailed guidance on modifying the architecture and loss functions, please refer to our documentation at https://docs.ultralytics.com. 🛠️

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐