search

Yujun-Shi / DragDiffusion

[CVPR2024, Highlight] Official code for DragDiffusion
https://yujun-shi.github.io/projects/dragdiffusion.html
Apache License 2.0
1.13k stars 82 forks source link

Question about Motion Supervision and Point Tracking #70

Open tnarek opened 1 month ago

tnarek commented 1 month ago

Hello,

Thanks for sharing your great work!

I have 2 questions regarding motion supervision:

  1. Why do you normalize the direction vector $d_i$ in the loss (equation 3)? Can't we directly optimize for matching the position $g_i$ itself? Is it for making the latent optimization more gradual? If so, how important is it?
  2. Maybe related to the previous question -- why is the point tracking step necessary? I see that in equation 3 you are taking the target feature $sg(F_q(\hat{z}^k_t))$ from the optimized latent, which of course requires $q$ to be updated in the next steps by point tracking. But why can't the target feature be taken from the original latent, without having to update $q$?