search

facebookresearch / co-tracker

CoTracker is a model for tracking any point (pixel) on a video.
https://co-tracker.github.io/
Other
2.72k stars 195 forks source link

Point tracking with different frame initializations #95

Closed Davidyao99 closed 2 months ago

Davidyao99 commented 3 months ago

Lets say I have a video with 20 frames. I am hoping to track some points that are initialized in frame 5, and some points that are initialized in frame 15. Naively, I can find the tractlets separately in frame 5 and frame 15 and concatenate the results together. However, this does not take advantage of the correlation between the points from frame 5 and frame 15.

Is it possible for me to obtain tracklets with initializations from different frames, while still being able to exploit the correlation between them? Thank you so much!

nikitakaraevv commented 2 months ago

Hi @Davidyao99, you can just initialise two different query points at frames 5 and 15 and the model will take care of all the correlations because it's trained to track from different frames. A query points is defined as follows: (t,x,y), where t is the frame number from which we start tracking, and x and y are the corresponding coordinates.

Davidyao99 commented 2 months ago

@nikitakaraevv I see, that works. Thank you so much!