search

SysCV / sam-pt

SAM-PT: Extending SAM to zero-shot video segmentation with point-based tracking.
https://arxiv.org/abs/2307.01197
Apache License 2.0
950 stars 60 forks source link

Real-time integration #2

Closed mertikinci closed 1 year ago

mertikinci commented 1 year ago

You are not mentioning any time performance results in your paper. Can we use it for segmenting real time ? Are there any fps rates results of your runs or how much time does it takes per frame with your setup ?

Thanks for your great work, best wishes,

m43 commented 1 year ago

Thank you for your interest in our work. Compared to state-of-the-art trackers such as XMem, SAM-based trackers currently do not run in real-time as their bottleneck lies in calling SAM's costly image encoder for each frame independently. On top of this, our method also introduces less costly point tracking. We acknowledge the importance of real-time processing for many applications and will come back to you once we have final and comprehensive inference times.

m43 commented 1 year ago

We added FPS results in the tables here, reported for running on an NVIDIA RTX A6000