search

google-deepmind / tapnet

Tracking Any Point (TAP)
https://deepmind-tapir.github.io/blogpost.html
Apache License 2.0
1.26k stars 119 forks source link

Comparison between TAP-Net and PIPs #3

Closed amitabe closed 1 year ago

amitabe commented 1 year ago

Hi,

You only mention that PIPs is a concurrent work to TAP-Vid/Net.

Are you planning to show any comparisons between these 2 works?

yangyi02 commented 1 year ago

We are working on it. Thanks for your patience.

HarryHsing commented 1 year ago

We are working on it. Thanks for your patience.

Hi, I hope this message finds you well. I was wondering if any progress has been made on this matter. Thank you!

cdoersch commented 1 year ago

The numbers have been added to the most recent version of our arxiv paper

HarryHsing commented 1 year ago

The numbers have been added to the most recent version of our arxiv paper

Thank you very much for the update! Will you release the evaluation code of PIPs on Kinetics, Kubric, DAVIS, and RGB-Stacking? I am really looking forward to it, thank you!

m43 commented 1 year ago

I've got PIPS results that are close to the ones reported in the updated arxiv paper using this somewhat refactored fork of PIPs.

My numbers are in the last two rows: image

However, my RAFT numbers do not match the reported ones, except on the DAVIS subset.

If interested, here are some entry points:

  1. Batched PIPS forward pass for chained trajectory prediction
  2. Evaluation for loop
  3. Documentation on running the evaluation
  4. PIPs environment setup
cdoersch commented 1 year ago

Sorry, we don't have plans to release the code we wrote to run PIPs on TAP-Vid. I feel it would belong in the PIPs repository anyway, as there's no PIPs code here.

Also, our approach to running PIPs is impractically slow: the chaining algorithm means we can essentially run only a single point at a time. We just hacked the chain_demo script from PIPs to do it and then waited. The overall code we needed to write was not very large.

The actual code to evaluate the output of PIPs, of course, is a part of this repository, and depends only on Numpy.

HarryHsing commented 1 year ago

I've got PIPS results that are close to the ones reported in the updated arxiv paper using this somewhat refactored fork of PIPs.

My numbers are in the last two rows: image

However, my RAFT numbers do not match the reported ones, except on the DAVIS subset.

If interested, here are some entry points:

  1. Batched PIPS forward pass for chained trajectory prediction
  2. Evaluation for loop
  3. Documentation on running the evaluation
  4. PIPs environment setup

Really appreciate it! Thanks!