search

wmuron / motpy

Library for tracking-by-detection multi object tracking implemented in python
MIT License
510 stars 60 forks source link

Retrieve detection score (and metadata) from Track #9

Closed gswebspace closed 3 years ago

gswebspace commented 3 years ago

Hi,

The score of a Detection can be provided as input. How can it be retrieved from the Track objects ( i.e the return value of active_tracks )?

For context, since detected boxes usually have a score/confidence value it needs to remain associated with the Detection/box when it takes the shape of a Track (i.e uuid gets added). This associated score has to pass through the tracker otherwise this information is lost and is not available for any downstream processing. The intent is to be able keep the score (or any metadata) associated with the Detection and retrieve it from the output Track.

Let me know if there is a way to do it currently ? and if it can be considered for implementation ?

wmuron commented 3 years ago

To be honest, the library is still in early development stage and although quite reliable, it might miss some of the implementations. That's why input detection accepts confidence score, but it is not used in matching or track update step (I assumed that the library user will provide already score-filtered detections to the multi object tracker, so no score-based filtering is applied when calling multi_object_tracker.step(...).

Also, the tracks retrieved from the tracker do not necessarily come from a single detection; think of it as a observed state of the actual object, where the detection is just a prediction of the object position used to update the state. If you need some kind of estimation of confidence for a given track, and you are okay with it being calculated purely from matched detection confidences, then you can use e.g. exponential moving average to update track's confidence score. But at the moment it will require forking library on your end and modifying Tracker.update() method.

gswebspace commented 3 years ago

To be honest, the library is still in early development stage and although quite reliable, it might miss some of the implementations.

I think it is an amazing library and It is quite reliable. 👍

I assumed that the library user will provide already score-filtered detections to the multi object tracker...

I agree with this. The input detections I am providing is filtered based on score already and I guess that is fine. My use case was mostly to store the score with the tracked object for later inspection/debugging to deal with any object-detection issues(if they arise). Having the score as part of the output of the tracker could help in tuning, but it is not a mandatory requirement for the tracker. There can be other ways to debug such issues.

If you need some kind of estimation of confidence for a given track...

I guess it is not really required. I don't have a solid use-case for Track confidence values right now. The object detection confidence values seem to be sufficient for filtering.

So let's not worry about the score. This issue can be closed for now. Thanks!

wmuron commented 3 years ago

@gswebspace Thanks, it's great to hear :)

Regarding the track score, I've decided to add this features anyway, because it seems right that input detections with specified confidence score do impact the output tracks confidences in some way. I've used score update with exponential moving average operation. If that approach works for you, feel free to use track.score value.