search

chenhsuanlin / inverse-compositional-STN

Inverse Compositional Spatial Transformer Networks :performing_arts: (CVPR 2017 oral)
MIT License
318 stars 63 forks source link

what is opt.refMtrx? #11

Closed jiangwei221 closed 6 years ago

jiangwei221 commented 6 years ago

In the IC-STN code(pytorch), in file, there are following 2 lines of code:

opt.refMtrx = warp.fit(Xsrc=opt.canon4pts,Xdst=opt.image4pts)
opt.refMtrx = np.eye(3).astype(np.float32)

If I commented the second line out, the performance dropped significantly and the network become harder to train.

However, the tensorflow version is using the results from 'warp.fit', and works fine.

What is the reason that for pytorch version, you are using np.eye(3), instead of warp.fit?

chenhsuanlin commented 6 years ago

The warped grids in Pytorch are in the canonical coordinate system (i.e. normalized within [-1,1]^2), whereas that in Tensorflow are in the image coordinate system. This is because the built-in function torch.nn.functional.grid_sample in Pytorch assumes that the regular grids are within [-1,1]^2. I've removed the first line to avoid confusion, thanks for pointing it out.