Shuaib11-Github
commented
7 months ago Hello Glenn, can you provide a simple example code on how to do this. I need to calculate MOTA and MOTP metrics. But for that they should be in same format.
My ground truth file is a .txt file with frame no, Object IDs, Bounding boxes
And the tracking has for some frame has objects IDs as None and for others it has detected and bounding boxes are in tensors for each object detected. For tracking how do can we obtain frame number
This is for the 300th frame and some frames have bounding boxes but no Object IDs For the 300th frame it has detected 23 objects Object IDs: tensor([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.]) Bounding Boxes: tensor([[1746.5852, 532.9240, 1919.6624, 752.6738], [ 671.1702, 433.9525, 746.4346, 495.2292], [ 631.0151, 388.8817, 681.9429, 427.3249], [ 750.9922, 409.3558, 809.0884, 456.6566], [1128.0063, 430.8089, 1202.2113, 484.1173], [1794.8990, 446.9329, 1918.4865, 507.2599], [1163.6973, 349.8423, 1202.3220, 381.4899], [1065.2070, 413.3709, 1131.9819, 464.2481], [ 852.6096, 404.4958, 911.0833, 446.3598], [1215.1777, 381.8876, 1265.1753, 421.5816], [1368.8850, 399.9973, 1424.5498, 440.3634], [ 957.9658, 404.4436, 1029.3635, 465.9520], [1518.6152, 334.2286, 1562.8381, 368.8765], [1609.2129, 333.2341, 1690.6794, 460.3948], [1027.0447, 340.4250, 1060.4711, 368.0705], [ 609.4783, 363.4427, 644.3919, 386.9360], [1131.8442, 349.2545, 1160.5519, 374.2233], [ 972.8117, 342.3387, 1005.2701, 366.1255], [ 908.2639, 302.0847, 956.2980, 336.6221], [1579.6724, 410.7565, 1613.2292, 459.0648], [1202.4021, 317.7359, 1228.8746, 340.8892], [ 761.5058, 364.0521, 793.8737, 388.1438], [ 842.4190, 361.6234, 886.5723, 391.3627]])
I have a ground truth data for 300 frames with frame_no, object_id, bounding boxes. ground truth data looks like this in a .txt file
41 1 952.0 0.0 79.0 14.0 42 1 954.7936639291984 0.0 83.82589528209907 14.867768595041323 44 1 961.9401867331467 0.0 80.44983453823076 14.258238809086198 50 1 969.075503083659 0.0 83.46678106945829 14.803607257496001 51 1 966.6347111485035 0.0 92.04233446195347 16.424031424653666 52 1 965.5744302572999 0.0 94.57271250223188 16.980587070598872 53 1 964.526253119986 0.0 98.34157491788552 17.83726246673933 ....
On Sat, 17 Feb 2024, 1:04 am Glenn Jocher, @.***> wrote:
@Shuaib11-Github https://github.com/Shuaib11-Github hello! It looks like you're on the right track with your tracking and validation efforts. For calculating MOTA and MOTP, you'll need to ensure that your detections and ground truth annotations are correctly paired by frame and object ID. Here's a simplified approach to help you:
- Make sure your ground truth and detection data are synchronized by frame number.
- Use the motmetrics library to create an accumulator and update it with the IoU matrix for each frame.
- After processing all frames, compute the MOTA and MOTP metrics using the accumulator.
Here's a snippet to guide you:
Initialize MOT metrics accumulatoracc = mm.MOTAccumulator(auto_id=True)
Loop through each framefor frame_number in range(start_frame, end_frame):
# Get ground truth and detection data for the current frame gt_data = ground_truth_data.get(frame_number, []) det_data = detection_data.get(frame_number, []) # Extract IDs and bounding boxes gt_ids, gt_boxes = zip(*gt_data) if gt_data else ([], []) det_ids, det_boxes = zip(*det_data) if det_data else ([], []) # Calculate IoU matrix iou_matrix = mm.distances.iou_matrix(gt_boxes, det_boxes, max_iou=0.5) # Update the accumulator acc.update( gt_ids, det_ids, iou_matrix )Compute MOTA and MOTPsummary = mh.compute(acc, metrics=['mota', 'motp'], name='acc')print(summary)
For the IndexError you're encountering, it seems there might be a mismatch in the dimensions of your IDs and bounding boxes. Double-check that the lengths of gt_ids and det_ids match the number of rows in your iou_matrix.
Regarding the frame number, if you're using model.track(), the frame number is typically the index of the frame in the video or stream. If object_ids is None for some frames, it means no detections were made in that frame. You can handle this by checking if det_ids is not None before updating the accumulator.
I hope this helps! If you have further questions or run into more issues, feel free to reach out. Good luck with your tracking project! 👍🏼
— Reply to this email directly, view it on GitHub https://github.com/ultralytics/ultralytics/issues/8252#issuecomment-1949206807, or unsubscribe https://github.com/notifications/unsubscribe-auth/AONNA2A3Q2774BDYRZUKKSTYT6YF5AVCNFSM6AAAAABDMMORPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBZGIYDMOBQG4 . You are receiving this because you were mentioned.Message ID: @.***>
glenn-jocher
github-actions[bot]
Search before asking
Question
I have for tracking for a single output as below, Object IDs: tensor([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23.]) Bounding Boxes: tensor([[1746.5852, 532.9240, 1919.6624, 752.6738], [ 671.1702, 433.9525, 746.4346, 495.2292], [ 631.0151, 388.8817, 681.9429, 427.3249], [ 750.9922, 409.3558, 809.0884, 456.6566], [1128.0063, 430.8089, 1202.2113, 484.1173], [1794.8990, 446.9329, 1918.4865, 507.2599], [1163.6973, 349.8423, 1202.3220, 381.4899], [1065.2070, 413.3709, 1131.9819, 464.2481], [ 852.6096, 404.4958, 911.0833, 446.3598], [1215.1777, 381.8876, 1265.1753, 421.5816], [1368.8850, 399.9973, 1424.5498, 440.3634], [ 957.9658, 404.4436, 1029.3635, 465.9520], [1518.6152, 334.2286, 1562.8381, 368.8765], [1609.2129, 333.2341, 1690.6794, 460.3948], [1027.0447, 340.4250, 1060.4711, 368.0705], [ 609.4783, 363.4427, 644.3919, 386.9360], [1131.8442, 349.2545, 1160.5519, 374.2233], [ 972.8117, 342.3387, 1005.2701, 366.1255], [ 908.2639, 302.0847, 956.2980, 336.6221], [1579.6724, 410.7565, 1613.2292, 459.0648], [1202.4021, 317.7359, 1228.8746, 340.8892], [ 761.5058, 364.0521, 793.8737, 388.1438], [ 842.4190, 361.6234, 886.5723, 391.3627]])
For ground truth the file looks like this Frame_number, object_id, bounding boxes 41 34710 952.0 0.0 79.0 14.0
but I have 300 such instances for which I need to calculate iou_matrix and MOTA and MOTP metrics. I am having difficulty in doing this. I don't know how do I calculate MOTA and MOTP for the above, here is the code
from ultralytics import YOLO import motmetrics as mm import os import numpy as np
def parse_ground_truth(file_path): ground_truth = {} with open(file_path, 'r') as file: for line in file: parts = line.strip().split() frame_number, obj_id, x_min, y_min, x_max, y_max = int(parts[0]), int(parts[1]), float(parts[2]), float(parts[3]), float(parts[4]), float(parts[5]) bbox = (x_min, y_min, x_max, y_max)
Load the YOLO model
model = YOLO('/home/rmarri/speed-estimation/bdd-yolo-finetune/runs/detect/train24/weights/best.pt')
Directories
video_dir = "/home/rmarri/speed-estimation/bdd-yolo-finetune/60fpsVidoes" # Update this path ground_truth_dir = "/home/rmarri/speed-estimation/bdd-yolo-finetune/60fpsvideolabels" # Update this path
Initialize MOT metrics
mh = mm.metrics.create() print("MOT initialized")
def calculate_iou(boxA, boxB):
Calculate the coordinates of the intersection rectangle
for video_file in os.listdir(video_dir): if not video_file.endswith(".MOV"): continue # Skip non-video files
Compute metrics after processing all frames
mh = mm.metrics.create() summary = mh.compute(acc, metrics=mm.metrics.motchallenge_metrics, name='acc') print(summary)
Additional
I have a ground truth data with frame_no, object_id, bounding boxes. ground truth data looks like this 41 1 952.0 0.0 79.0 14.0 42 1 954.7936639291984 0.0 83.82589528209907 14.867768595041323 44 1 961.9401867331467 0.0 80.44983453823076 14.258238809086198 50 1 969.075503083659 0.0 83.46678106945829 14.803607257496001 51 1 966.6347111485035 0.0 92.04233446195347 16.424031424653666 52 1 965.5744302572999 0.0 94.57271250223188 16.980587070598872 53 1 964.526253119986 0.0 98.34157491788552 17.83726246673933
I am getting this GT IDs: (334710,) # groundtruth DET IDs: [ 1] # detection GT Boxes: [[ 952 0 79 14]] DET Boxes: [[ 0 2.9919 1295.9 1076.3]] IoU Matrix: [[ 0]] GT IDs: (334710,) DET IDs: [ 1] GT Boxes: [[ 954.79 0 83.826 14.868]] DET Boxes: [[ 0 2.0868 1276.7 1073.6]] IoU Matrix: [[ 0]] GT IDs: (334710,) DET IDs: None GT Boxes: [[ 961.94 0 80.45 14.258]] DET Boxes: [[ 0 0 1386.8 1066.7] [ 18.699 453.94 73.065 504.86] [ 75.047 454.59 134.77 506.24]] IoU Matrix: [[ 0] [ -0] [ -0]] Traceback (most recent call last): File "botsort2.py", line 108, in
acc.update(
File "/home/rmarri/anaconda3/envs/yoloft/lib/python3.8/site-packages/motmetrics/mot.py", line 181, in update
dists = np.atleast_2d(dists).astype(float).reshape(oids.shape[0], hids.shape[0]).copy()
IndexError: tuple index out of range
And for tracking I don't know how to get frame_no. For some of the frames the object_ids is None. I want to calculate MOTA and MOTP metrics for ground truth and tracking.