Open AFallDay opened 4 weeks ago
@AFallDay hey there! Integrating the calculation of Precision (P), Recall (R), Average Precision (AP), and Mean Average Precision (mAP) from val.py
into detect.py
involves a bit of work, as these metrics are typically calculated during model validation rather than single image detections.
Here’s a brief rundown on how to start this:
Import Required Functions: You’ll need to import functions that calculate these metrics from val.py
. This includes handling True Positives, False Positives, etc.
Modify detect.py
: In the detect.py
, after detections are made by the model, you should use the imported functions to calculate these metrics. Keep in mind that detecting and validating are different in terms of processing (e.g., you usually have ground truth data in validation).
Integrate Calculation after Detections: After each detection in detect.py
, calculate the metrics based on the ground truths of your data (you’ll need to provide this somehow).
Here's a sample code snippet for the run
function in detect.py
after modifying:
# Assuming your model outputs and ground truth labels are obtained
true_pos, false_pos, false_neg = some_metric_calculation(pred_boxes, true_boxes)
# Calculate Precision, Recall
precision = true_pos / (true_pos + false_pos)
recall = true_pos / (true_pos + false_neg)
# Store or print your calculated metrics
print(f'Precision: {precision}, Recall: {recall}')
For complete step-by-step integration, you need to maintain your modifications aligned with how detections are being processed in your detect.py
, especially handling batches of data if you’re doing so.
Remember, this is non-trivial and requires good understanding of how detection and validation work in YOLOv5. Good luck! 😊 If you face specific issues while performing these steps, feel free to raise them with detailed code snippets and errors if any.
@glenn-jocher Can I just import additional labels for ground truths in detect.py to calculate P, R, AP, MAP metrics?
Hey there! Yes, you can import additional labels for ground truths in detect.py
to calculate metrics like Precision (P), Recall (R), Average Precision (AP), and Mean Average Precision (mAP). However, you'll need to ensure that these labels are correctly aligned with the detections made by the model for accurate metric calculation. You'll also need to modify the detection script to compare these ground truth labels against the model's predictions and then compute the metrics based on this comparison. Keep in mind that this requires careful handling of data formats and the logic for metric calculation. Good luck! 😊
Search before asking
Question
May I ask yolov5 how to port the method of calculating P, R, AP, MAP in val.py to adapt to detect.py, what code need to be packed? and where should I add or change it? Please answer this question, thanks!
The code for detect.py is as follows:
import argparse import csv import os import platform import sys from pathlib import Path
import torch
FILE = Path(file).resolve() ROOT = FILE.parents[0] # YOLOv5 root directory if str(ROOT) not in sys.path: sys.path.append(str(ROOT)) # add ROOT to PATH ROOT = Path(os.path.relpath(ROOT, Path.cwd())) # relative
from ultralytics.utils.plotting import Annotator, colors, save_one_box
from models.common import DetectMultiBackend from utils.dataloaders import IMG_FORMATS, VID_FORMATS, LoadImages, LoadScreenshots, LoadStreams from utils.general import (LOGGER, Profile, check_file, check_img_size, check_imshow, check_requirements, colorstr, cv2, increment_path, non_max_suppression, print_args, scale_boxes, strip_optimizer, xyxy2xywh) from utils.torch_utils import select_device, smart_inference_mode
@smart_inference_mode() def run( weights='D:\yolov5/runs/train\exp16\weights/best.pt', # model path or triton URL source='D:/dataset/images/val', # file/dir/URL/glob/screen/0(webcam) data='D:/yolov5/VOC.yaml', # dataset.yaml path imgsz=(640, 640), # inference size (height, width) conf_thres=0.25, # confidence threshold iou_thres=0.45, # NMS IOU threshold max_det=1000, # maximum detections per image device='', # cuda device, i.e. 0 or 0,1,2,3 or cpu view_img=False, # show results save_txt=False, # save results to *.txt save_csv=False, # save results in CSV format save_conf=False, # save confidences in --save-txt labels save_crop=False, # save cropped prediction boxes nosave=False, # do not save images/videos classes=None, # filter by class: --class 0, or --class 0 2 3 agnostic_nms=False, # class-agnostic NMS augment=False, # augmented inference visualize=False, # visualize features update=False, # update all models project=ROOT / 'runs/detect', # save results to project/name name='exp', # save results to project/name exist_ok=False, # existing project/name ok, do not increment line_thickness=3, # bounding box thickness (pixels) hide_labels=False, # hide labels hide_conf=False, # hide confidences half=False, # use FP16 half-precision inference dnn=False, # use OpenCV DNN for ONNX inference vid_stride=1, # video frame-rate stride ): source = str(source) save_img = not nosave and not source.endswith('.txt') # save inference images is_file = Path(source).suffix[1:] in (IMG_FORMATS + VID_FORMATS) is_url = source.lower().startswith(('rtsp://', 'rtmp://', 'http://', 'https://')) webcam = source.isnumeric() or source.endswith('.streams') or (is_url and not is_file) screenshot = source.lower().startswith('screen') if is_url and is_file: source = check_file(source) # download
def parse_opt(): parser = argparse.ArgumentParser()
parser.add_argument('--weights', nargs='+', type=str, default= 'C:/Users\hp\Desktop\AF\guipian\model\exp10\weights/best.pt', help='model path or triton URL')
def main(opt): check_requirements(ROOT / 'requirements.txt', exclude=('tensorboard', 'thop')) run(**vars(opt))
if name == 'main': opt = parse_opt() main(opt)
The code for val.py is as follows: import argparse import json import os import subprocess import sys from pathlib import Path
import numpy as np import torch from tqdm import tqdm
FILE = Path(file).resolve() ROOT = FILE.parents[0] # YOLOv5 root directory if str(ROOT) not in sys.path: sys.path.append(str(ROOT)) # add ROOT to PATH ROOT = Path(os.path.relpath(ROOT, Path.cwd())) # relative
from models.common import DetectMultiBackend from utils.callbacks import Callbacks from utils.dataloaders import create_dataloader from utils.general import (LOGGER, TQDM_BAR_FORMAT, Profile, check_dataset, check_img_size, check_requirements, check_yaml, coco80_to_coco91_class, colorstr, increment_path, non_max_suppression, print_args, scale_boxes, xywh2xyxy, xyxy2xywh) from utils.metrics import ConfusionMatrix, ap_per_class, box_iou from utils.plots import output_to_target, plot_images, plot_val_study from utils.torch_utils import select_device, smart_inference_mode from PIL import Image, ImageDraw
def save_one_txt(predn, save_conf, shape, file):
Save one txt result
def save_one_json(predn, jdict, path, class_map):
Save one JSON result {"image_id": 42, "category_id": 18, "bbox": [258.15, 41.29, 348.26, 243.78], "score": 0.236}
def process_batch(detections, labels, iouv): """ Return correct prediction matrix Arguments: detections (array[N, 6]), x1, y1, x2, y2, conf, class labels (array[M, 5]), class, x1, y1, x2, y2 Returns: correct (array[N, 10]), for 10 IoU levels """ correct = np.zeros((detections.shape[0], iouv.shape[0])).astype(bool) iou = box_iou(labels[:, 1:], detections[:, :4]) correct_class = labels[:, 0:1] == detections[:, 5] for i in range(len(iouv)): x = torch.where((iou >= iouv[i]) & correct_class) # IoU > threshold and classes match if x[0].shape[0]: matches = torch.cat((torch.stack(x, 1), iou[x[0], x[1]][:, None]), 1).cpu().numpy() # [label, detect, iou] if x[0].shape[0] > 1: matches = matches[matches[:, 2].argsort()[::-1]] matches = matches[np.unique(matches[:, 1], return_index=True)[1]]
matches = matches[matches[:, 2].argsort()[::-1]]
@smart_inference_mode() def run( data= 'D:/yolov5/VOC.yaml', weights='D:\guipian\model\exp16\weights/best.pt', # model.pt path(s) batch_size=1, # batch size imgsz=640, # inference size (pixels) conf_thres=0.001, # confidence threshold iou_thres=0.6, # NMS IoU threshold max_det=300, # maximum detections per image task='val', # train, val, test, speed or study device='', # cuda device, i.e. 0 or 0,1,2,3 or cpu workers=0, # max dataloader workers (per RANK in DDP mode) single_cls=False, # treat as single-class dataset augment=False, # augmented inference verbose=False, # verbose output save_txt=False, # save results to .txt save_hybrid=False, # save label+prediction hybrid results to .txt save_conf=False, # save confidences in --save-txt labels save_json=False, # save a COCO-JSON results file project=ROOT / 'runs/val', # save to project/name name='exp', # save to project/name exist_ok=False, # existing project/name ok, do not increment half=True, # use FP16 half-precision inference dnn=False, # use OpenCV DNN for ONNX inference model=None, dataloader=None, save_dir=Path(''), plots=True, callbacks=Callbacks(), compute_loss=None, ):
Initialize/load model and set device
def parse_opt(): parser = argparse.ArgumentParser() parser.add_argument('--data', type=str, default='D:/yolov5/VOC.yaml', help='dataset.yaml path') parser.add_argument('--weights', nargs='+', type=str, default='D:\yolov5/runs\exp_val\exp9/weights/best.pt', help='model path(s)') parser.add_argument('--batch-size', type=int, default=1, help='batch size') parser.add_argument('--imgsz', '--img', '--img-size', type=int, default=640, help='inference size (pixels)') parser.add_argument('--conf-thres', type=float, default=0.25, help='confidence threshold') parser.add_argument('--iou-thres', type=float, default=0.5, help='NMS IoU threshold') parser.add_argument('--max-det', type=int, default=300, help='maximum detections per image') parser.add_argument('--task', default='val', help='train, val, test, speed or study') parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') parser.add_argument('--workers', type=int, default=8, help='max dataloader workers (per RANK in DDP mode)') parser.add_argument('--single-cls', action='store_true', help='treat as single-class dataset') parser.add_argument('--augment', action='store_true', help='augmented inference') parser.add_argument('--verbose', default=True,action='store_true', help='report mAP by class') parser.add_argument('--save-txt', action='store_true', help='save results to .txt') parser.add_argument('--save-hybrid', action='store_true', help='save label+prediction hybrid results to .txt') parser.add_argument('--save-conf', action='store_true', help='save confidences in --save-txt labels') parser.add_argument('--save-json', action='store_true', help='save a COCO-JSON results file') parser.add_argument('--project', default=ROOT / 'runs/exp_val_map', help='save to project/name') parser.add_argument('--name', default='exp', help='save to project/name') parser.add_argument('--exist-ok', action='store_true', help='existing project/name ok, do not increment') parser.add_argument('--half', action='store_true', help='use FP16 half-precision inference') parser.add_argument('--dnn', action='store_true', help='use OpenCV DNN for ONNX inference') opt = parser.parse_args() opt.data = check_yaml(opt.data) # check YAML opt.save_json |= opt.data.endswith('coco.yaml') opt.save_txt |= opt.save_hybrid print_args(vars(opt)) return opt
def main(opt): check_requirements(ROOT / 'requirements.txt', exclude=('tensorboard', 'thop'))
if name == 'main': opt = parse_opt() main(opt)
Additional
No response