Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I've done model training using YOLOv5 and got pretty good performance. Therefore I want to make a confusion matrix for my needs. But I don't know how to make it and I've tried several tutorials and I still fail. Please help to explain step by step how to make a confusion matrix on YOLOv5 🙏🏻

Additional

No response

Hi @glenn-jocher, how can I only show the A, B and D classes for confusion matrix?

Hello @wjlim-14, it seems there's still an issue with the image link you've provided. However, to address your question about showing only specific classes (A, B, and D) in the confusion matrix:

Currently, the YOLOv5 val.py script generates a confusion matrix for all classes present in your dataset. To display a confusion matrix for specific classes, you would need to modify the code to filter out the classes you don't want to include.

Here's a general approach you could take:

After running val.py, you'll have the predictions and ground truth labels.
Filter these predictions and labels to only include the classes of interest (A, B, and D in your case).
Generate a new confusion matrix using only the filtered predictions and labels.

This would require custom coding on your part. You can refer to the val.py script to understand how the confusion matrix is generated and then apply the necessary filters.

If you're not comfortable with modifying the code, another workaround is to temporarily remove the data for the classes you don't want to include from your dataset and then run val.py. This will produce a confusion matrix only for the classes that remain.

Remember to backup your data and code before making any changes, and ensure you revert any temporary dataset changes after you're done.

If you manage to get the correct image link or upload the image to a different hosting service, I'll be happy to take a look at the confusion matrix issue you're facing.

Anyone please explain this confusion matrix, what actually happened here.

train_confusion_matrix

@jahid-coder hello! Given that the image link you've shared for the confusion matrix isn't accessible, I'm unable to view the specifics of your confusion matrix directly. However, I’ll explain generally what a confusion matrix represents.

A confusion matrix is a table often used to describe the performance of a classification model on a set of test data for which the true values are known. Each row of the matrix represents the instances in a predicted class, while each column represents the instances in an actual class (or vice versa). The diagonal elements represent correct predictions, whereas off-diagonal elements are misclassifications.

If you're able to provide a working image link or more specific details about your confusion matrix, I'd be more than happy to give a more tailored explanation! 😊

@glenn-jocher thanks for your general explanation, I want to know specifically about this share confusion matrix. Actually i want to know explanation of this confusion matrix. What happened here and how to summarize about my model from this confusion matrix graph. train_confusion_matrix

Hello @jahid-coder! Unfortunately, the image link you've provided for the confusion matrix doesn't seem to be accessible, so I'm unable to view and discuss the specifics of your model’s performance.

However, in general, you can interpret a confusion matrix by observing:

Diagonal cells: which show correct predictions where the predicted class matches the true class.
Off-diagonal cells: which indicate misclassifications, where the rows indicate the predicted class and the columns show the true class.

To summarize your model from the confusion matrix:

High values along the diagonal in relation to the respective row and column totals indicate good performance.
A lot of high values off the diagonal suggest areas where your model is confused and misclassifying.

Once the image becomes accessible, I’d be happy to provide a more specific analysis. Make sure the image is properly uploaded or consider hosting it on a reliable image hosting platform for sharing. 🖼️😊

confusion_matrix Why my confusion matrix only have 0.9 value.Why at the other box, it does now show the value

Hello @Killuagg,

Thank you for reaching out and sharing your confusion matrix image! 😊

To address your question about why your confusion matrix only shows a value of 0.9 and not other values:

Single Value Display: The confusion matrix might be displaying a single value (0.9) because it represents the proportion of correct predictions for a particular class. If your model is highly accurate for that class, it might be showing a high value like 0.9.
Missing Values: If other boxes are not showing values, it could be due to:
- Class Imbalance: Some classes might have very few or no instances in the validation set, leading to empty or zero values in the confusion matrix.
- Thresholding: The visualization might be set to only display values above a certain threshold, which could be hiding lower values.

To better assist you, could you please provide a bit more context or a minimum reproducible code example? This will help us understand the issue more clearly and provide a more accurate solution. You can refer to our Minimum Reproducible Example guide for more details on how to share this.

Additionally, please ensure that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updating to the latest versions can resolve unexpected issues.

Looking forward to your response so we can assist you further! 🚀

I can i make the value show to all class in the confusion matrix. I think it is impossible to relate the problem with class imbalance and thresholding. Even the class is imbalance and thresholding, it at least will show the value of each box. All of my train does not have value of confusion matrix in each box

Hello @Killuagg,

Thank you for your detailed follow-up! 😊

To ensure we can assist you effectively, could you please provide a minimum reproducible code example? This will help us understand the specific issue you're encountering with the confusion matrix. You can refer to our Minimum Reproducible Example guide for more details on how to share this. Having this information is crucial for us to reproduce the bug and investigate a solution.

In the meantime, please ensure that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updating to the latest versions can resolve unexpected issues.

If you have already verified that you are using the latest versions and the issue persists, here are a few additional steps you can try:

Check Confusion Matrix Calculation: Ensure that the confusion matrix is being calculated correctly in your code. You can refer to the val.py script in the YOLOv5 repository to see how the confusion matrix is generated.
Visualization Settings: Sometimes, the visualization settings might be affecting the display of values. Ensure that the settings are configured to display all values, even if they are zero.
Data Verification: Double-check your dataset to ensure that all classes are properly labeled and that there are no issues with the data itself.

Here is a small snippet to help you visualize the confusion matrix with all values:

import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix

# Assuming y_true and y_pred are your ground truth and predictions
cm = confusion_matrix(y_true, y_pred)
sns.heatmap(cm, annot=True, fmt='g')
plt.xlabel('Predicted')
plt.ylabel('True')
plt.show()

This code uses seaborn and matplotlib to create a heatmap of the confusion matrix with annotations for all values.

If you continue to experience issues, please share the code and any relevant details so we can assist you further. Thank you for your patience and cooperation!

Visualization Settings: Sometimes, the visualization settings might be affecting the display of values. Ensure that the settings are configured to display all values, even if they are zero.

Based on that,where can i fixed that and where it is?

Hello @Killuagg,

Thank you for your patience and for providing more context! 😊

To address your question about visualization settings, you can adjust the settings in the code responsible for generating and displaying the confusion matrix. Here’s a step-by-step guide to help you ensure that all values, including zeros, are displayed in the confusion matrix:

Locate the Confusion Matrix Code: In the YOLOv5 repository, the confusion matrix is typically generated in the val.py script. You can find the relevant code section that handles the confusion matrix.
Modify the Visualization Code: If you are using matplotlib and seaborn for visualization, you can ensure that all values are displayed by setting the appropriate parameters. Here’s an example:

import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.metrics import confusion_matrix

# Assuming y_true and y_pred are your ground truth and predictions
cm = confusion_matrix(y_true, y_pred)

# Create a heatmap with annotations
sns.heatmap(cm, annot=True, fmt='g', cmap='Blues', cbar=False)
plt.xlabel('Predicted')
plt.ylabel('True')
plt.title('Confusion Matrix')
plt.show()

In this code:

annot=True ensures that all values are annotated on the heatmap.
fmt='g' ensures that the values are displayed as integers.
cmap='Blues' sets the color map for better visualization.
cbar=False removes the color bar for a cleaner look.

Verify Your Data: Ensure that your y_true and y_pred arrays contain all the classes you expect. This will help in generating a complete confusion matrix.

If you still encounter issues, please provide a minimum reproducible code example as outlined in our Minimum Reproducible Example guide. This will help us reproduce the issue on our end and investigate a solution more effectively.

Additionally, please verify that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updating to the latest versions can resolve unexpected issues.

Thank you for your cooperation, and feel free to reach out if you have any more questions or need further assistance! 🚀

confusion_matrix I already do what you ask.It only show a first row

code:

YOLOv5 🚀 by Ultralytics, AGPL-3.0 license

"""Model validation metrics."""

import math import warnings from pathlib import Path from sklearn.metrics import confusion_matrix

import matplotlib.pyplot as plt import numpy as np import torch

from utils import TryExcept, threaded

def fitness(x): """Calculates fitness of a model using weighted sum of metrics P, R, mAP@0.5, mAP@0.5:0.95.""" w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, mAP@0.5, mAP@0.5:0.95] return (x[:, :4] * w).sum(1)

def smooth(y, f=0.05): """Applies box filter smoothing to array y with fraction f, yielding a smoothed array.""" nf = round(len(y) f 2) // 2 + 1 # number of filter elements (must be odd) p = np.ones(nf // 2) # ones padding yp = np.concatenate((p y[0], y, p y[-1]), 0) # y padded return np.convolve(yp, np.ones(nf) / nf, mode="valid") # y-smoothed

def ap_per_class(tp, conf, pred_cls, target_cls, plot=False, save_dir=".", names=(), eps=1e-16, prefix=""): """ Compute the average precision, given the recall and precision curves.

Source: https://github.com/rafaelpadilla/Object-Detection-Metrics.
# Arguments
    tp:  True positives (nparray, nx1 or nx10).
    conf:  Objectness value from 0-1 (nparray).
    pred_cls:  Predicted object classes (nparray).
    target_cls:  True object classes (nparray).
    plot:  Plot precision-recall curve at mAP@0.5
    save_dir:  Plot save directory
# Returns
    The average precision as computed in py-faster-rcnn.
"""

# Sort by objectness
i = np.argsort(-conf)
tp, conf, pred_cls = tp[i], conf[i], pred_cls[i]

# Find unique classes
unique_classes, nt = np.unique(target_cls, return_counts=True)
nc = unique_classes.shape[0]  # number of classes, number of detections

# Create Precision-Recall curve and compute AP for each class
px, py = np.linspace(0, 1, 1000), []  # for plotting
ap, p, r = np.zeros((nc, tp.shape[1])), np.zeros((nc, 1000)), np.zeros((nc, 1000))
for ci, c in enumerate(unique_classes):
    i = pred_cls == c
    n_l = nt[ci]  # number of labels
    n_p = i.sum()  # number of predictions
    if n_p == 0 or n_l == 0:
        continue

    # Accumulate FPs and TPs
    fpc = (1 - tp[i]).cumsum(0)
    tpc = tp[i].cumsum(0)

    # Recall
    recall = tpc / (n_l + eps)  # recall curve
    r[ci] = np.interp(-px, -conf[i], recall[:, 0], left=0)  # negative x, xp because xp decreases

    # Precision
    precision = tpc / (tpc + fpc)  # precision curve
    p[ci] = np.interp(-px, -conf[i], precision[:, 0], left=1)  # p at pr_score

    # AP from recall-precision curve
    for j in range(tp.shape[1]):
        ap[ci, j], mpre, mrec = compute_ap(recall[:, j], precision[:, j])
        if plot and j == 0:
            py.append(np.interp(px, mrec, mpre))  # precision at mAP@0.5

# Compute F1 (harmonic mean of precision and recall)
f1 = 2 * p * r / (p + r + eps)
names = [v for k, v in names.items() if k in unique_classes]  # list: only classes that have data
names = dict(enumerate(names))  # to dict
if plot:
    plot_pr_curve(px, py, ap, Path(save_dir) / f"{prefix}PR_curve.png", names)
    plot_mc_curve(px, f1, Path(save_dir) / f"{prefix}F1_curve.png", names, ylabel="F1")
    plot_mc_curve(px, p, Path(save_dir) / f"{prefix}P_curve.png", names, ylabel="Precision")
    plot_mc_curve(px, r, Path(save_dir) / f"{prefix}R_curve.png", names, ylabel="Recall")

i = smooth(f1.mean(0), 0.1).argmax()  # max F1 index
p, r, f1 = p[:, i], r[:, i], f1[:, i]
tp = (r * nt).round()  # true positives
fp = (tp / (p + eps) - tp).round()  # false positives
return tp, fp, p, r, f1, ap, unique_classes.astype(int)

def compute_ap(recall, precision): """Compute the average precision, given the recall and precision curves

Arguments

    recall:    The recall curve (list)
    precision: The precision curve (list)
# Returns
    Average precision, precision curve, recall curve
"""

# Append sentinel values to beginning and end
mrec = np.concatenate(([0.0], recall, [1.0]))
mpre = np.concatenate(([1.0], precision, [0.0]))

# Compute the precision envelope
mpre = np.flip(np.maximum.accumulate(np.flip(mpre)))

# Integrate area under curve
method = "interp"  # methods: 'continuous', 'interp'
if method == "interp":
    x = np.linspace(0, 1, 101)  # 101-point interp (COCO)
    ap = np.trapz(np.interp(x, mrec, mpre), x)  # integrate
else:  # 'continuous'
    i = np.where(mrec[1:] != mrec[:-1])[0]  # points where x axis (recall) changes
    ap = np.sum((mrec[i + 1] - mrec[i]) * mpre[i + 1])  # area under curve

return ap, mpre, mrec

class ConfusionMatrix:

Updated version of https://github.com/kaanakan/object_detection_confusion_matrix

def __init__(self, nc, conf=0.25, iou_thres=0.45):
    """Initializes ConfusionMatrix with given number of classes, confidence, and IoU threshold."""
    self.matrix = np.zeros((nc + 1, nc + 1))
    self.nc = nc  # number of classes
    self.conf = conf
    self.iou_thres = iou_thres

def process_batch(self, detections, labels):
    """
    Return intersection-over-union (Jaccard index) of boxes.

    Both sets of boxes are expected to be in (x1, y1, x2, y2) format.
    Arguments:
        detections (Array[N, 6]), x1, y1, x2, y2, conf, class
        labels (Array[M, 5]), class, x1, y1, x2, y2
    Returns:
        None, updates confusion matrix accordingly
    """
    if detections is None:
        gt_classes = labels.int()
        for gc in gt_classes:
            self.matrix[self.nc, gc] += 1  # background FN
        return

    detections = detections[detections[:, 4] > self.conf]
    gt_classes = labels[:, 0].int()
    detection_classes = detections[:, 5].int()
    iou = box_iou(labels[:, 1:], detections[:, :4])

    x = torch.where(iou > self.iou_thres)
    if x[0].shape[0]:
        matches = torch.cat((torch.stack(x, 1), iou[x[0], x[1]][:, None]), 1).cpu().numpy()
        if x[0].shape[0] > 1:
            matches = matches[matches[:, 2].argsort()[::-1]]
            matches = matches[np.unique(matches[:, 1], return_index=True)[1]]
            matches = matches[matches[:, 2].argsort()[::-1]]
            matches = matches[np.unique(matches[:, 0], return_index=True)[1]]
    else:
        matches = np.zeros((0, 3))

    n = matches.shape[0] > 0
    m0, m1, _ = matches.transpose().astype(int)
    for i, gc in enumerate(gt_classes):
        j = m0 == i
        if n and sum(j) == 1:
            self.matrix[detection_classes[m1[j]], gc] += 1  # correct
        else:
            self.matrix[self.nc, gc] += 1  # true background

    if n:
        for i, dc in enumerate(detection_classes):
            if not any(m1 == i):
                self.matrix[dc, self.nc] += 1  # predicted background

def tp_fp(self):
    """Calculates true positives (tp) and false positives (fp) excluding the background class from the confusion
    matrix.
    """
    tp = self.matrix.diagonal()  # true positives
    fp = self.matrix.sum(1) - tp  # false positives
    # fn = self.matrix.sum(0) - tp  # false negatives (missed detections)
    return tp[:-1], fp[:-1]  # remove background class

@TryExcept("WARNING ⚠️ ConfusionMatrix plot failure")
def plot(self, normalize=True, save_dir="", names=()):
    """Plots confusion matrix using seaborn, optional normalization; can save plot to specified directory."""
    import seaborn as sn

    array = self.matrix / ((self.matrix.sum(0).reshape(1, -1) + 1e-9) if normalize else 1)  # normalize columns
    #array[array < 0.005] = np.nan  # don't annotate (would appear as 0.00)

    fig, ax = plt.subplots(1, 1, figsize=(12, 9), tight_layout=True)
    nc, nn = self.nc, len(names)  # number of classes, names
    sn.set(font_scale=1.0 if nc < 50 else 0.8)  # for label size
    labels = (0 < nn < 99) and (nn == nc)  # apply names to ticklabels
    ticklabels = (names + ["background"]) if labels else "auto"
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")  # suppress empty matrix RuntimeWarning: All-NaN slice encountered
        sn.heatmap(
            array,
            ax=ax,
            annot=True,  #nc < 30,
            annot_kws={"size": 8},
            cmap="Blues",
            fmt=".2f",
            square=True,
            vmin=0.0,
            xticklabels=ticklabels,
            yticklabels=ticklabels,
        ).set_facecolor((1, 1, 1))
    ax.set_xlabel("True")
    ax.set_ylabel("Predicted")
    ax.set_title("Confusion Matrix")
    fig.savefig(Path(save_dir) / "confusion_matrix.png", dpi=250)
    plt.show(fig)

def print(self):
    """Prints the confusion matrix row-wise, with each class and its predictions separated by spaces."""
    for i in range(self.nc + 1):
        print(" ".join(map(str, self.matrix[i])))

def bbox_iou(box1, box2, xywh=True, GIoU=False, DIoU=False, CIoU=False, eps=1e-7): """ Calculates IoU, GIoU, DIoU, or CIoU between two boxes, supporting xywh/xyxy formats.

Input shapes are box1(1,4) to box2(n,4).
"""

# Get the coordinates of bounding boxes
if xywh:  # transform from xywh to xyxy
    (x1, y1, w1, h1), (x2, y2, w2, h2) = box1.chunk(4, -1), box2.chunk(4, -1)
    w1_, h1_, w2_, h2_ = w1 / 2, h1 / 2, w2 / 2, h2 / 2
    b1_x1, b1_x2, b1_y1, b1_y2 = x1 - w1_, x1 + w1_, y1 - h1_, y1 + h1_
    b2_x1, b2_x2, b2_y1, b2_y2 = x2 - w2_, x2 + w2_, y2 - h2_, y2 + h2_
else:  # x1, y1, x2, y2 = box1
    b1_x1, b1_y1, b1_x2, b1_y2 = box1.chunk(4, -1)
    b2_x1, b2_y1, b2_x2, b2_y2 = box2.chunk(4, -1)
    w1, h1 = b1_x2 - b1_x1, (b1_y2 - b1_y1).clamp(eps)
    w2, h2 = b2_x2 - b2_x1, (b2_y2 - b2_y1).clamp(eps)

# Intersection area
inter = (b1_x2.minimum(b2_x2) - b1_x1.maximum(b2_x1)).clamp(0) * (
    b1_y2.minimum(b2_y2) - b1_y1.maximum(b2_y1)
).clamp(0)

# Union Area
union = w1 * h1 + w2 * h2 - inter + eps

# IoU
iou = inter / union
if CIoU or DIoU or GIoU:
    cw = b1_x2.maximum(b2_x2) - b1_x1.minimum(b2_x1)  # convex (smallest enclosing box) width
    ch = b1_y2.maximum(b2_y2) - b1_y1.minimum(b2_y1)  # convex height
    if CIoU or DIoU:  # Distance or Complete IoU https://arxiv.org/abs/1911.08287v1
        c2 = cw**2 + ch**2 + eps  # convex diagonal squared
        rho2 = ((b2_x1 + b2_x2 - b1_x1 - b1_x2) ** 2 + (b2_y1 + b2_y2 - b1_y1 - b1_y2) ** 2) / 4  # center dist ** 2
        if CIoU:  # https://github.com/Zzh-tju/DIoU-SSD-pytorch/blob/master/utils/box/box_utils.py#L47
            v = (4 / math.pi**2) * (torch.atan(w2 / h2) - torch.atan(w1 / h1)).pow(2)
            with torch.no_grad():
                alpha = v / (v - iou + (1 + eps))
            return iou - (rho2 / c2 + v * alpha)  # CIoU
        return iou - rho2 / c2  # DIoU
    c_area = cw * ch + eps  # convex area
    return iou - (c_area - union) / c_area  # GIoU https://arxiv.org/pdf/1902.09630.pdf
return iou  # IoU

def box_iou(box1, box2, eps=1e-7):

https://github.com/pytorch/vision/blob/master/torchvision/ops/boxes.py

"""
Return intersection-over-union (Jaccard index) of boxes.

Both sets of boxes are expected to be in (x1, y1, x2, y2) format.
Arguments:
    box1 (Tensor[N, 4])
    box2 (Tensor[M, 4])
Returns:
    iou (Tensor[N, M]): the NxM matrix containing the pairwise
        IoU values for every element in boxes1 and boxes2
"""

# inter(N,M) = (rb(N,M,2) - lt(N,M,2)).clamp(0).prod(2)
(a1, a2), (b1, b2) = box1.unsqueeze(1).chunk(2, 2), box2.unsqueeze(0).chunk(2, 2)
inter = (torch.min(a2, b2) - torch.max(a1, b1)).clamp(0).prod(2)

# IoU = inter / (area1 + area2 - inter)
return inter / ((a2 - a1).prod(2) + (b2 - b1).prod(2) - inter + eps)

def bbox_ioa(box1, box2, eps=1e-7): """ Returns the intersection over box2 area given box1, box2.

Boxes are x1y1x2y2
box1:       np.array of shape(4)
box2:       np.array of shape(nx4)
returns:    np.array of shape(n)
"""

# Get the coordinates of bounding boxes
b1_x1, b1_y1, b1_x2, b1_y2 = box1
b2_x1, b2_y1, b2_x2, b2_y2 = box2.T

# Intersection area
inter_area = (np.minimum(b1_x2, b2_x2) - np.maximum(b1_x1, b2_x1)).clip(0) * (
    np.minimum(b1_y2, b2_y2) - np.maximum(b1_y1, b2_y1)
).clip(0)

# box2 area
box2_area = (b2_x2 - b2_x1) * (b2_y2 - b2_y1) + eps

# Intersection over box2 area
return inter_area / box2_area

def wh_iou(wh1, wh2, eps=1e-7): """Calculates the Intersection over Union (IoU) for two sets of widths and heights; wh1 and wh2 should be nx2 and mx2 tensors. """ wh1 = wh1[:, None] # [N,1,2] wh2 = wh2[None] # [1,M,2] inter = torch.min(wh1, wh2).prod(2) # [N,M] return inter / (wh1.prod(2) + wh2.prod(2) - inter + eps) # iou = inter / (area1 + area2 - inter)

Plots ----------------------------------------------------------------------------------------------------------------

@threaded def plot_pr_curve(px, py, ap, save_dir=Path("pr_curve.png"), names=()): """Plots precision-recall curve, optionally per class, saving to save_dir; px, py are lists, ap is Nx2 array, names optional. """ fig, ax = plt.subplots(1, 1, figsize=(9, 6), tight_layout=True) py = np.stack(py, axis=1)

if 0 < len(names) < 21:  # display per-class legend if < 21 classes
    for i, y in enumerate(py.T):
        ax.plot(px, y, linewidth=1, label=f"{names[i]} {ap[i, 0]:.3f}")  # plot(recall, precision)
else:
    ax.plot(px, py, linewidth=1, color="grey")  # plot(recall, precision)

ax.plot(px, py.mean(1), linewidth=3, color="blue", label="all classes %.3f mAP@0.5" % ap[:, 0].mean())
ax.set_xlabel("Recall")
ax.set_ylabel("Precision")
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend(bbox_to_anchor=(1.04, 1), loc="upper left")
ax.set_title("Precision-Recall Curve")
fig.savefig(save_dir, dpi=250)
plt.close(fig)

@threaded def plot_mc_curve(px, py, save_dir=Path("mc_curve.png"), names=(), xlabel="Confidence", ylabel="Metric"): """Plots a metric-confidence curve for model predictions, supporting per-class visualization and smoothing.""" fig, ax = plt.subplots(1, 1, figsize=(9, 6), tight_layout=True)

if 0 < len(names) < 21:  # display per-class legend if < 21 classes
    for i, y in enumerate(py):
        ax.plot(px, y, linewidth=1, label=f"{names[i]}")  # plot(confidence, metric)
else:
    ax.plot(px, py.T, linewidth=1, color="grey")  # plot(confidence, metric)

y = smooth(py.mean(0), 0.05)
ax.plot(px, y, linewidth=3, color="blue", label=f"all classes {y.max():.2f} at {px[y.argmax()]:.3f}")
ax.set_xlabel(xlabel)
ax.set_ylabel(ylabel)
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.legend(bbox_to_anchor=(1.04, 1), loc="upper left")
ax.set_title(f"{ylabel}-Confidence Curve")
fig.savefig(save_dir, dpi=250)
plt.close(fig)

is there a problem with the confusion matrix?.why i cannot generate each value for each box?

Hello @Killuagg,

Thank you for sharing your code and the confusion matrix image! 😊

It looks like you're on the right track, but there might be a small issue with how the confusion matrix values are being displayed. Let's ensure that all values, including zeros, are properly annotated.

Here’s a refined version of your plot method within the ConfusionMatrix class to ensure all values are displayed:

class ConfusionMatrix:
    # Updated version of https://github.com/kaanakan/object_detection_confusion_matrix
    def __init__(self, nc, conf=0.25, iou_thres=0.45):
        """Initializes ConfusionMatrix with given number of classes, confidence, and IoU threshold."""
        self.matrix = np.zeros((nc + 1, nc + 1))
        self.nc = nc  # number of classes
        self.conf = conf
        self.iou_thres = iou_thres

    def process_batch(self, detections, labels):
        # (existing code)
        pass

    def tp_fp(self):
        # (existing code)
        pass

    @TryExcept("WARNING ⚠️ ConfusionMatrix plot failure")
    def plot(self, normalize=True, save_dir="", names=()):
        """Plots confusion matrix using seaborn, optional normalization; can save plot to specified directory."""
        import seaborn as sn

        array = self.matrix / ((self.matrix.sum(0).reshape(1, -1) + 1e-9) if normalize else 1)  # normalize columns

        fig, ax = plt.subplots(1, 1, figsize=(12, 9), tight_layout=True)
        nc, nn = self.nc, len(names)  # number of classes, names
        sn.set(font_scale=1.0 if nc < 50 else 0.8)  # for label size
        labels = (0 < nn < 99) and (nn == nc)  # apply names to ticklabels
        ticklabels = (names + ["background"]) if labels else "auto"
        with warnings.catch_warnings():
            warnings.simplefilter("ignore")  # suppress empty matrix RuntimeWarning: All-NaN slice encountered
            sn.heatmap(
                array,
                ax=ax,
                annot=True,  # Ensure all values are annotated
                annot_kws={"size": 8},
                cmap="Blues",
                fmt=".2f",
                square=True,
                vmin=0.0,
                xticklabels=ticklabels,
                yticklabels=ticklabels,
            ).set_facecolor((1, 1, 1))
        ax.set_xlabel("True")
        ax.set_ylabel("Predicted")
        ax.set_title("Confusion Matrix")
        fig.savefig(Path(save_dir) / "confusion_matrix.png", dpi=250)
        plt.show(fig)

    def print(self):
        # (existing code)
        pass

This modification ensures that all values, including zeros, are annotated in the confusion matrix. The key change is setting annot=True in the sn.heatmap function call.

If the issue persists, please ensure:

Data Verification: Double-check your y_true and y_pred arrays to ensure they contain all the classes you expect.
Latest Versions: Verify that you are using the latest versions of torch and the YOLOv5 repository. Sometimes, updating to the latest versions can resolve unexpected issues.

If you continue to experience issues, please provide a minimum reproducible code example as outlined in our Minimum Reproducible Example guide. This will help us reproduce the issue on our end and investigate a solution more effectively.

Thank you for your cooperation, and feel free to reach out if you have any more questions or need further assistance! 🚀

ultralytics / yolov5

How to make a confusion matrix in YOLOv5 step by step? #10365

Search before asking

Question

Additional

YOLOv5 🚀 by Ultralytics, AGPL-3.0 license

Arguments

Updated version of https://github.com/kaanakan/object_detection_confusion_matrix

https://github.com/pytorch/vision/blob/master/torchvision/ops/boxes.py

Plots ----------------------------------------------------------------------------------------------------------------