Smoother Fails Due to Missing tracker_id in Detections

YoungjaeDev commented 5 months ago

Search before asking

[X] I have searched the Supervision issues and found no similar bug report.

Bug

I'm experiencing an issue with the supervision library when using the smoother for better visualization. While my model is successfully detecting objects, it seems that tracker_id is not being assigned to the detections (maybe due to threshold). This causes the smoother to fail with the following error:

IndexError: index 0 is out of bounds for axis 0 with size 0

Here are the details of the error:

Traceback (most recent call last):
  File "app.py", line 110, in <module>
    main()
  File "app.py", line 104, in main
    process_video((video_path, xml_dir_path, save_video_dir_path, use_cut_video, parse_gt))
  File "app.py", line 51, in process_video
    results = KisaModel.run(frame, frame_number)
  File "/home/aicads/workspace/kisa_cctv_cert/models/model.py", line 67, in run
    results[key] = tracker.process(frame_number, detections)
  File "/home/aicads/workspace/kisa_cctv_cert/models/intrusion.py", line 41, in process
    detections_filtered = self.smoother.update_with_detections(detections=detections_filtered)
  File "/home/aicads/miniconda3/envs/kisa_cert/lib/python3.8/site-packages/supervision/detection/tools/smoother.py", line 80, in update_with_detections
    tracker_id = detections.tracker_id[detection_idx]
IndexError: index 0 is out of bounds for axis 0 with size 0

It appears that the tracker_id is not being generated, which leads to the smoother failing. Shouldn't there be an exception handling or a check to ensure that tracker_id is available before attempting to use it?

Here is a snippet of the detections object:

Detections(xyxy=array([[1256.5, 72.485, 1279.9, 126.03]], dtype=float32), mask=None, confidence=array([0.42017], dtype=float32), class_id=array([0]), tracker_id=array([], dtype=int64), data={'class_name': array(['person'], dtype='<U6')})

As you can see, the tracker_id array is empty.

Please advise on how to handle this situation or if there is a bug fix in progress for this issue.

Environment

Supervision 0.20.0

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

[ ] Yes I'd like to help by submitting a PR!

YoungjaeDev commented 5 months ago

It's not experiencing the issue with supervision 0.19.0, so which one is the bug?

SkalskiP commented 5 months ago

Hi, @YoungjaeDev 👋🏻 do you have a tracker plugged into your pipeline?

YoungjaeDev commented 5 months ago

Hi, @YoungjaeDev 👋🏻 do you have a tracker plugged into your pipeline?

Yes, of course I'm using it. It's the same code, it works for 0.19.0, but not 0.20.0.

SkalskiP commented 5 months ago

@YoungjaeDev, can you try to see if supervision==0.21.0rc5 works?

SMUEric1127 commented 5 months ago

Any ideas? I'm having the same issue too

LinasKo commented 5 months ago

Hi @SMUEric1127 :wave:

Can you try pip install supervision==0.21.0rc5 too?

YoungjaeDev commented 5 months ago

@LinasKo @SkalskiP

supervision==0.21.0rc5 seems to smooth out the behavior

What was the cause?

LinasKo commented 5 months ago

That's good to hear :)

I know we've had issues with tracker_id generation. If you're curious, feel free to compare the files in: https://github.com/roboflow/supervision/tree/develop/supervision/tracker/byte_tracker. Relevant tags are 0.20.0 versus 0.21.0rc5.

I'll close the issue - it seems to be resolved :slightly_smiling_face:

roboflow / supervision