search

qianqianwang68 / omnimotion

Apache License 2.0
2.07k stars 121 forks source link

Affine Transformation #27

Open amitabe opened 9 months ago

amitabe commented 9 months ago

Hi,

I see that you are applying a learned affine transformation for all (x,y) in the same frame and depth before feeding (x,y,z) into the deformation MLP.

I didn't find any information about it in the paper. Can you explain the motivation for having this transformation?

qianqianwang68 commented 9 months ago

The motivation for this transformation is to improve the ability to model large displacement in the 2D space, which we found to help with challenging scenes with large motion or complex ordering and help the network converge faster initially. However, we found that the average numerical number on the benchmark is not affected by this transformation, which means it might hurt some other sequences.