Where and When Non-Maximum Suppression(NMS) happens?

[tbensap18]

Search before asking

• [X] I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hello all, I humbly ask for some assistance in understanding the NMS method as I am new to YOLO and would like to further my knowledge. I know that NMS is used with most, if not all YOLO versions, so I'm thinking it's used here as well, so I have the following questions. Any help is appreciated.

Question

Is NMS used in YOLOv8 as well? If yes, where and when in the YOLOv8 architecture does NMS happen? #

Thanks in advance

Additional

No response

[github-actions[bot]]

👋 Hello @tbensap18, thank you for your interest in YOLOv8 🚀! We recommend a visit to the YOLOv8 Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Install

Pip install the ultralytics package including all requirements in a Python>=3.7 environment with PyTorch>=1.7.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@tbensap18 hello! Yes, like previous versions of YOLO, YOLOv8 also uses Non-Maximum Suppression (NMS). In YOLOv8, NMS is applied after the individual bounding box predictions are made by the model. Specifically, it is applied after the model predicts the locations, sizes and class probabilities of various objects present in an input image, and generates multiple bounding boxes for each object. Since some of these bounding boxes might overlap, NMS is used to remove the ones with lower confidence scores and retain only the best ones. Thus, NMS helps to ensure that each object is detected only once and that higher quality predictions are retained. I hope that helps, let us know if you have further questions!

[tbensap18]

@tbensap18 hello! Yes, like previous versions of YOLO, YOLOv8 also uses Non-Maximum Suppression (NMS). In YOLOv8, NMS is applied after the individual bounding box predictions are made by the model. Specifically, it is applied after the model predicts the locations, sizes and class probabilities of various objects present in an input image, and generates multiple bounding boxes for each object. Since some of these bounding boxes might overlap, NMS is used to remove the ones with lower confidence scores and retain only the best ones. Thus, NMS helps to ensure that each object is detected only once and that higher quality predictions are retained. I hope that helps, let us know if you have further questions!

@glenn-jocher Thank you very much for the response. I have one last question. I found in an article that YOLOv5 and YOLOv8, along with other YOLO versions use DIoU__nms. Is this true for YOLOv5 & YOLOv8? If it is, then what is the difference of this DIoU_nms with the original nms. Thanks again

[glenn-jocher]

@tbensap18, that's a great question. Yes, you are correct that YOLOv5 and YOLOv8 use DIoU_nms. This is an updated version of the original NMS algorithm which takes into account the distance between the predicted bounding boxes and the ground truth bounding boxes. The DIoU term stands for "distance IoU" and it considers the overlap between two bounding boxes in addition to the distances between them. Unlike traditional NMS which looks only at the overlap between two bounding boxes, DIoU_nms considers both the overlap as well as their distance and ensures that the predictions with the highest overall quality are kept. This often results in more accurate detections and better overlapping box suppression when compared to the original NMS. I hope that helps!

[tbensap18]

@glenn-jocher Thanks for all the responses. I believe that my questions have all been answered on this matter. Wish you the best of luck in your future.

[glenn-jocher]

@tbensap18 thank you for your kind words and support! We are always here to help, so feel free to reach out if you have any further questions in the future. Best of luck to you as well!

[Zynade]

@tbensap18, that's a great question. Yes, you are correct that YOLOv5 and YOLOv8 use DIoU_nms. This is an updated version of the original NMS algorithm which takes into account the distance between the predicted bounding boxes and the ground truth bounding boxes. The DIoU term stands for "distance IoU" and it considers the overlap between two bounding boxes in addition to the distances between them. Unlike traditional NMS which looks only at the overlap between two bounding boxes, DIoU_nms considers both the overlap as well as their distance and ensures that the predictions with the highest overall quality are kept. This often results in more accurate detections and better overlapping box suppression when compared to the original NMS. I hope that helps!

Hi @glenn-jocher, I have a question regarding this. You mentioned that DIoU is used in the NMS algorithm to consider the distance between the predicted bbox and ground truth bbox. However, the NMS algorithm is only run during inference, and in inference, there are no ground truth labels. What am I missing?

[glenn-jocher]

@Zynade, you've touched on an important detail, and I appreciate the opportunity to clarify.

Indeed, during inference, we do not have ground truth bounding boxes available as we do during training. The mention of ground truth was a miscommunication on my part—my apologies for any confusion caused.

To clarify, DIoU-NMS during inference does not compare predicted bounding boxes to ground truth, but rather to each other. It uses the Intersection over Union (IoU) to measure the overlap between predicted boxes. The 'D' in DIoU stands for distance, but it refers to the distance between the center points of the different predicted boxes. It penalizes boxes that are far away from each other in terms of their center point distance while rewarding those that are closer, which can be especially helpful in situations where objects are close together.

This means DIoU-NMS can help in reducing redundancy and preserving the most accurate bounding box when there are multiple overlapping detections, without any reference to ground truth labels.

Thank you for catching that inconsistency, and I hope this explanation helps!

[dinhthihuyen]

Hi @glenn-jocher, As I see in code, YOLOv8 using IoU-NMS during inference instead of DIoU-NMS: def postprocess(self, preds): """Apply Non-maximum suppression to prediction outputs.""" return ops.non_max_suppression(preds, self.args.conf, self.args.iou, labels=self.lb, multi_label=True, agnostic=self.args.single_cls, max_det=self.args.max_det) Where is DIoU implemented? I have another question: As I understand, with bbox detected, we have postprocess method: Firstly, it removes all bbox with conf_score below conf_thres. Then, it uses IoU-NMS to remove all bboxes that have IoU with highest score bbox higher than iou threshold for NMS. Is my understanding true? Thanks so much for your support!

[glenn-jocher]

@dinhthihuyen hello,

Indeed, YOLOv8 applies traditional IoU-based NMS during inference by default. The implementation you've referenced is the standard NMS, where it filters out detections below the confidence threshold and suppresses overlapping boxes based on the IoU threshold you mentioned.

DIoU-NMS is an alternative method that can be used in place of traditional IoU-based NMS. This is implemented separately, and it takes into account both the IoU and the distances between the centers of bounding boxes.

Your understanding of the postprocessing steps is correct. First, detections below a certain confidence score are discarded. Afterwards, IoU-NMS is applied to suppress redundant bounding boxes, keeping only the ones with the highest confidence scores that meet the IoU criteria.

Let me know if you need further clarification on any of these points!

[AphroDatalyst]

hey @glenn-jocher thanks for ur explanation. may we set a fixed size for each bbox (like for specific classes have special w,h box size) ? may we implement it in NMS ? i wanna have a specific cropped bbox image size not to resize detected boxes.

[glenn-jocher]

@AphroDatalyst hi there! 👋

Setting a fixed size for bounding boxes isn't directly supported in the NMS process, as NMS is designed to select the best bounding box based on overlap and confidence scores. However, you can certainly post-process the NMS output to enforce fixed sizes for specific classes.

Here's a quick example of how you might adjust the bounding box size after NMS:

for i, det in enumerate(detections):  # Loop through detections
    if det[5] == class_id:  # Check if detection class matches your target class
        x_center, y_center = (det[0] + det[2]) / 2, (det[1] + det[3]) / 2
        detections[i][0] = x_center - fixed_width / 2  # Set left x
        detections[i][1] = y_center - fixed_height / 2  # Set top y
        detections[i][2] = x_center + fixed_width / 2  # Set right x
        detections[i][3] = y_center + fixed_height / 2  # Set bottom y

Just replace class_id, fixed_width, and fixed_height with your specific class ID and desired dimensions. This code snippet adjusts the bounding box to have a fixed width and height while keeping the center the same.

Hope this helps! Let me know if you have any other questions. 😊

[rohitsawant780]

When i do predictions using trained yolov8n model sometimes for same object i get two bounding boxex , how can i take only one from this . I am detecting the cracks from the tongue . Sometimes i got two predictions for same crack by overlapping boxex so i want to take only one from this , and also if it detect two seperate cracks then i want to take both . so if the boxex are overlapping then i want to take one from that . So how can i do this ? How can i use nms while predictions? do i have to implement my logic or yolov8n is providing its parameter from nms while predictions. I am using this code model = YOLO(model_path) img = foreground_image results = model(img) for r in results: boxes=r.boxes for box in boxes: x1, y1, x2, y2 = box.xyxy[0] x1, y1, x2, y2 = int(x1), int(y1), int(x2), int(y2) final_photo= img[int(y1):int(y2), int(x1):int(x2)] if view == 1: cv2.imshow("Cropped Image", final_photo) cv2.waitKey(0) cv2.destroyAllWindows() cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2) .

[glenn-jocher]

Hey there! 👋 It sounds like you're encountering overlapped detections, which is quite common in object detection tasks! YOLOv8 already uses NMS (Non-Maximum Suppression) during predictions to handle such cases, aiming to keep only the most confident detection when boxes overlap significantly.

If you're noticing overlapped boxes even after this, you might want to adjust the NMS threshold (how much overlap is allowed) to be stricter. Unfortunately, YOLOv8 doesn't expose an NMS threshold parameter directly at prediction time as in some other versions or frameworks.

However, you can fine-tune the confidence threshold to filter out less confident boxes, which indirectly contributes to reducing overlaps. Use the conf parameter to set the confidence level for detections:

model = YOLO(model_path, conf=0.25) # Adjust the 0.25 to your needs

For handling overlapped boxes manually, you may explore additional post-processing of results.boxes, but keep in mind this could get a bit involved.

Hope this helps, and best of luck with detecting those cracks! If you have any more questions, just let me know. 😊

[rohitsawant780]

how to adjust NMS threshold? in predict method there is iou paramter and agnostic_nms which is suitable for this ? I am getting two three boxes on same object that is crack , i want to keep only one from this . I am getting two three bounding boxes for one same crack with different confidene 0.49,.052,0.60 like this ,also i am getting different sepearte box for another crack so i want to keep only one box from the overlapping boxex and i also want to keep the box for the seperate crack and the confidence for the seperate box is 0.42 .So i want to keep for each seperate detected crack keep only one box . what can i do ?

[glenn-jocher]

Hey there! 👋 It sounds like adjusting the NMS threshold could help refine your detections. In the predict method of YOLOv8, the iou parameter directly influences NMS behavior by controlling how much overlap is allowed between boxes. Lowering iou will make the suppression more aggressive, meaning fewer overlapping boxes will be kept. Meanwhile, agnostic_nms doesn't adjust the threshold but dictates whether class labels are considered when suppressing duplicates; it's useful when you don't want to mix object classes.

For your specific case, lowering the iou might help. Try something like:

results = model(img, iou=0.3)  # Adjust the 0.3 as needed

This adjustment should help in keeping only one box per detected crack when boxes are overlapping, while still preserving boxes for separate cracks. Play around with the iou value to find the sweet spot for your application. Hope that helps! Let me know if you have more questions. 😊

[Krutarth-Bhatt]

@glenn-jocher Thanks for the advice on adjusting NMS as I was also looking for the same.

I had a followup question: Confidence Threshold will also affect the NMS correct? Because according to my understanding in the NMS algo, initially the bounding boxes with confidence < Confidence Threshold are removed.

[glenn-jocher]

Hey @Krutarth-Bhatt! You're absolutely right. The Confidence Threshold does indeed play a crucial role in the NMS process. Before NMS even kicks in, all detections with confidence scores below this threshold are filtered out. This means they won't be considered in the subsequent steps of NMS, where overlapping boxes are suppressed based on the iou parameter.

So, adjusting the Confidence Threshold can significantly impact which detections are ultimately considered and suppressed during NMS. Here’s a quick example of how you might set it:

results = model(img, conf=0.4)  # Adjust the 0.4 as needed

This helps in fine-tuning which detections to keep based on their confidence before NMS handles the overlaps. Hope this clears things up! 😊