search

google-deepmind / tapnet

Tracking Any Point (TAP)
https://deepmind-tapir.github.io/blogpost.html
Apache License 2.0
1.28k stars 120 forks source link

Is it possible to improve online demo performance with any hyperparameter changes? #60

Closed JoshMSmith44 closed 1 month ago

JoshMSmith44 commented 1 year ago

In some point tracking works, for example, temporal window size can be adjusted to improve performance. Are there any hyper-parameter changes from the default live_demo (causal model) that can improve tracking performance, even if it increases model runtime or memory use?

cdoersch commented 12 months ago

Sorry I missed this.

This model is trained end-to-end, so there's no simple way to trade off compute time for performance.

If your data does not contain any occlusions and objects move slowly, it may be possible to increase accuracy (and compute time) by replacing the TAP-Net-style global search with an initialization that uses the output from the prior frames. Feeding this directly into the refinement iterations should work fine. However, this would be a non-trivial code change and not something we directly support. If you try it, let us know how it goes.

cdoersch commented 1 month ago

Closing this due to inactivity.