search

qianqianwang68 / omnimotion

Apache License 2.0
2.12k stars 123 forks source link

test time optimization #2

Closed lzl2040 closed 1 year ago

lzl2040 commented 1 year ago

Why is it a test-time optimization method? I do not find something is adopted in the test stage.

rlee3359 commented 1 year ago

For each video, they optimize the model on all frames that can then be queried to track pixels through the video. Thus it is a "test-time" optimization method since there is no set of weights or something that works on all videos, you instead build the representation of the whole video and then you can do tracking. It's like a NeRF that requires a bunch of frames to make the implicit representation.

qianqianwang68 commented 1 year ago

@rlee3359 's explanation is correct. Thank you! Let me know if there is any other question.