search

thunil / TecoGAN

This repo contains source code and materials for the TEmporally COherent GAN SIGGRAPH project.
Apache License 2.0
5.95k stars 1.13k forks source link

Questions on your Optical Flow method #81

Closed l-oneil closed 3 years ago

l-oneil commented 3 years ago

Hi, thanks so much for pushing your full code to Github, it's fantastic work!

Just a couple of questions on the motion compensation you perform:

  1. In this line here why do you multiply your low resolution flow by 4, when upscaling by 4? gen_flow = upscale_four(gen_flow_lr*4.0)

  2. In this line here, you multiply the output of the tanh activation by 24.0 which you describe as "max velocity", am I correct in understanding this as the network being limited to compensate motion up to a displacement of +/- 24 pixels, but no further?

Kind Regards, Liam

itmo153277 commented 3 years ago

I am not related to the authors, but from what I can understand:

  1. The flow is just a coordinate offset map for warping. So when resolution is changed, naturally coordinates have to be scaled.
  2. Tanh has value range from -1 to 1, so by multiplying it by 24 it makes the maximum offset 24 pixels (for non-scaled images).
l-oneil commented 3 years ago

Thanks, that's what I thought too!