Inference with sequence of multiple images for a particular single pixel using co-tracker2

facebookresearch / co-tracker

CoTracker is a model for tracking any point (pixel) on a video.

https://co-tracker.github.io/

Other

2.52k stars 175 forks source link

Inference with sequence of multiple images for a particular single pixel using co-tracker2 #59

Open pknmax opened 5 months ago

pknmax commented 5 months ago

How to do inference with sequence of multiple images for a particular single pixel? Is there a way to select this pixel in first image?

nikitakaraevv commented 5 months ago

Hi @pknmax, please see the "Tracking manually selected points" section in the Colab Demo to see how to do it. Note that tracking individual points works best with support point grids , which are used by default in the CoTrackerPredictor class and discarded at the output.

pknmax commented 5 months ago

Hi @nikitakaraevv ; thanks for the reply. does the model manage to track the pixel if it has specular color (view dependent appearance) and changes colors after few frames?

nikitakaraevv commented 5 months ago

Hi @pknmax, the model works for such pixels in some cases, but not always.