search

tensorturtle / classy-sort-yolov5

Ready-to-use realtime multi-object tracker that works for any object category. YOLOv5 + SORT implementation.
GNU General Public License v3.0
110 stars 29 forks source link

Sort not helping when detector lost object #1

Closed OleksiiYeromenko closed 3 years ago

OleksiiYeromenko commented 3 years ago

I've compared results on the same video with pure Yolov5 and Classy-sort-yolov5 - result is the same. Whenever Yolov5 lost object behind an obstacle, your combined model also does not output object bounding boxes.

tensorturtle commented 3 years ago

SORT (Simple Online Realtime Tracker) does not have the ability to output bounding boxes in video frames (images) where the object being tracked is occluded (hidden behind something). However, SORT does not 'forget' the object. It keeps in its memory its last seen location, velocity, and acceleration (using a Kalman Filter), so that when the object is seen again after several frames, the detected object is re-identified as the same object.

You can try increasing --sort-max-age argument if you find that classy-sort-yolov5 is failing to re-identify correctly.

I think that it would be possible to modify classy-sort-yolov5 to output predicted bounding boxes for occluded objects. Please feel free to submit a pull request if you succeed in it.

OleksiiYeromenko commented 3 years ago

Thanks for the reply. Yes, I thought as SORT is based on Kalman Filter, it should be possible to predict bounding boxes for temporarily occluded objects. I will try to build a custom solution.

tensorturtle commented 3 years ago

I wish you the best of luck! Let me know if I can help.