search

facebookresearch / co-tracker

CoTracker is a model for tracking any point (pixel) on a video.
https://co-tracker.github.io/
Other
2.52k stars 174 forks source link

Training time of Co-tracker #21

Open memoiry opened 10 months ago

memoiry commented 10 months ago

Hi,

do we know how long it takes to finish 50,000 iterations with 32 V100 GPU?

Thanks!

nikitakaraevv commented 10 months ago

Hi @memoiry, for us, it takes around 40 hours with regular evaluations.

memoiry commented 10 months ago

Thanks! @nikitakaraevv , do you have any thoughts on how to do the windowed inference for realtime usage? I just thought a dumb “Window” approach where we windowed inference on every new frame, which might be time-consuming.

nikitakaraevv commented 10 months ago

Hi @memoiry, you could train a model with a sliding window of size 4. It is not much worse than the default window of size 8 (we have this experiment in the paper supplement). In this case, the lag will be only 1 frame (because the size of the overlap is 2 frames). For true real-time inference, you could train the model with a sliding window of size 2. This would eliminate lag, but we haven't tested it yet. :)