Tracking alongwith object detection

[jvidyad]

Hello, I have an issue with object tracking and object detection. I have a model doing object detection which returns bounding boxes, confidences and labels. I pass in the bounding boxes+confidences to tracker.update method of SORT and it outputs bounding boxes + object IDs. My question is, what is a reliable way to map the bounding boxes which this method output to the earlier bounding boxes so that I can create a mapping between the labels from object detection and the IDs from tracking using SORT

[jvidyad]

Hello, I also have an additional question. The bounding boxes output by the update method of Sort, are they for the current frame or a prediction for the next frame?

[jsvidyad]

Can someone help me with this?

[laclouis5]

Instantiate 1 Sort() tracker per label/class and feed each of them with boxes of that class.

[gn1024]

Can someone help me with this?

I've made the following 4 changes in sort.py and it's solved the same problem in my own project.

class KalmanBoxTracker(object):
    ...
    def __init__(self, bbox):
         ...
         self.original_id = bbox[5] # <--- add to keep track of original IDs

 def update(self, bbox):
        """
        Updates the state vector with observed bbox.
        """
        ...
        self.original_id = bbox[5]  # <--- update to keep track of original IDs
        self.kf.update(convert_bbox_to_z(bbox))

class Sort(object):
    ...
    def update(self, dets=np.empty((0, 6))):  # <--- change (0, 5) to (0, 6). Pass your detection IDs as 6th element in each list
    ...
    if (trk.time_since_update < 1) and (trk.hit_streak >= self.min_hits or self.frame_count <= self.min_hits):
         ret.append(np.concatenate((d, [trk.id + 1], [trk.original_id])).reshape(1, -1))  # <--- add [trk.original_id] to the returned set

[azharshaikh06]

@gn1024 by original id do you mean the object label or the first id?

[rgkannan676]

@gn1024 by original id do you mean the object label or the first id?

I think it is the object label id obtained after object detection, which can be used in the final results along with tracking details.