levan92 / deep_sort_realtime

A really more real-time adaptation of deep sort
MIT License
166 stars 51 forks source link

Question about embeds #30

Closed RoninHunter closed 2 years ago

RoninHunter commented 2 years ago

Do all forms of tracking require embeds?

I am asking because I am looking at your example below:

from deep_sort_realtime.deepsort_tracker import DeepSort
tracker = DeepSort(max_age=5)
bbs = object_detector.detect(frame) 
tracks = tracker.update_tracks(bbs, frame=frame) # bbs expected to be a list of detections, each in tuples of ( [left,top,w,h], confidence, detection_class )
for track in tracks:
    if not track.is_confirmed():
        continue
    track_id = track.track_id
    ltrb = track.to_ltrb()

Looking at your code I see all frames being funneled to a "generate_embeds()", however I don't see where the frame is given to the tracker similar to the above example.

levan92 commented 2 years ago

DeepSORT performs multi-object tracking based on: (1) bounding box coordinates, (2) appearance features. For more details of how this algorithm works, please take a look at the original author's paper. update_tracks expects to either already receive pre-computed appearance features (via any external module/model), or it expects to receive the raw frames in order to compute the appearance features using the default embedders as described here.

Looking at your code I see all frames being funneled to a "generate_embeds()", however I don't see where the frame is given to the tracker similar to the above example.

This "funnelling" only happens if embeds is not given. In the example you quoted, the frame is as I explained in this other issue.