search

chenhsuanlin / spatial-transformer-GAN

ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing :eyeglasses: (CVPR 2018)
MIT License
334 stars 72 forks source link

Help with understanding network #25

Closed guigo-gues closed 5 years ago

guigo-gues commented 5 years ago

Dear chenhsuanlin,

thank you for the paper and code. May I ask you what is the difference between your network and the one in https://cg.cs.tsinghua.edu.cn/papers/CGF-2018-video-stab.pdf? Apart from the decoder part.

Can you please suggest me some way to change your network so that it is similar to the one in the paper? I am trying to implement the network, however I think it is a bit confusing, with all the warps they perform

I would appreciate it if you could help me

Kind regards Gui

chenhsuanlin commented 5 years ago

I took a quick look at this paper. The main difference is that unlike ST-GAN which warps the input image multiple times, they predict a spatial transformation for each output feature of the same image, conditioned on the previous steady frames. It may be easier to first note the difference of the pipelines of this video stabilization network and ST-GAN or IC-STN, so you'll have a better idea of how to implement it by modifying my code. Hope this helps!

chenhsuanlin commented 5 years ago

Closing for now, please feel free to reopen if there are other issues.