search

SysCV / sam-pt

SAM-PT: Extending SAM to zero-shot video segmentation with point-based tracking.
https://arxiv.org/abs/2307.01197
Apache License 2.0
950 stars 60 forks source link

Disappearing tracked objects #18

Closed vilemil closed 11 months ago

vilemil commented 11 months ago

Hello,

First of all, congrats! Great work.

I run a few test and it has remarkable precision. Except one case: disappearing-reappearing objects, works really poorly (almost every case the tracking was lost) . I'm trying to track different, moving objects with moving camera. I haven't dug into your work in detail yet, but do you have any suggestions for that scenario? (more positive points for example?). I tested TAM also and it's working really well in this case(because of XMem), but has worse precision with small, blurry objects.

m43 commented 11 months ago

Hej, thanks for your interest! Re-identification can be very challenging for the point trackers. I'd suggest you first try replacing PIPS (i.e., the default point tracker) with the state-of-the-art CoTracker and see whether this helps. You can do so by adding the command line arguments model/point_tracker=cotracker model.positive_points_per_mask=16.

vilemil commented 11 months ago

Thank you! I will definitely check it out.