search

shleecs / Copy-and-Paste-Networks-for-Deep-Video-Inpainting

Official Pytorch implementation of Copy-and-Paste Networks for Deep Video Inpainting (ICCV 2019)
117 stars 18 forks source link

How to back-propagate to the alignment encoder and regressor? #4

Closed y60 closed 4 years ago

y60 commented 4 years ago

Thank you for releasing the codes of your great work! I am interested in how to back-propagate to the alignment encoder and regressor . Could you please give me details?

shleecs commented 4 years ago

Hello! Thank you for your interest for our works.

The alignment encoder and regressor are trained with self-supervised method.

Our alignment network extracts the affine matrix for aligning reference frame toward the target frame.

Our model computes the L_align loss between target frame and aligned reference frames(warped by affine matrix).

Furthermore, the other loss also affects the training the alignment networks, because our model is end-to-end deep neural networks,

You can find the details in our papers. Thank you!

y60 commented 4 years ago

Thank you very much for your detailed answer and it completely solved my question. (I did't know functional. grid_sample propagates the grad to the input grid. I am sorry for my misunderstanding.)