Spatial transformer network (STN) layer

[oarriaga]

Hi, I don't know if the STNs could be considered relevant enough to include them in this repo; however, I currently have an implementation of the STN layer in this gist. My original repository with a test case on the cluttered MNIST dataset can be found here. It is currently working with the tf backend (one can run the mnist_cluttered_example.ipynb). However, I recently moved everything into K tensors (in the K_tensors branch) but the meshgrid from theano is failing. Looking at the CONTRIBUTING.md it seems like you only accept PRs that work with both back-ends, is this the case? Also, if someone is interested in having a look at the failing K_meshgrid function in the theano backend and helping me fix it, it would be awesome.

Thank you.

[the-moliver]

Hi @oarriaga a STN would be great to include but we do need it to work on both backends. I looked at your code and it does seem fixable but you might want to do some refactoring. It seems T.mgrid[x, y] can't take the vectors you are passing it, the input has to look like T.mgrid[-1:1:.1,0:5:.1] so it may be easier to write K_meshgrid like:

# Tensorflow
def K_meshgrid(start1, end1, num1, start2, end2, num2):
    return tf.meshgrid(tf.linspace(start1, stop1, num1), tf.linspace(start2, stop2, num2))

# Theano
def K_meshgrid(start1, end1, num1, start2, end2, num2):
    step1 = ((stop1 - start1) / num1)
    step2 = ((stop2 - start2) / num2)
    return T.mgrid[start1:end1:step1, start2:end2:step2]

And then I think you can get rid of your linspace functions.

[tanmayb123]

I'd love to see an implementation of this - I really wonder why noone has done this yet!

[sallamander]

@oarriaga do you plan on pushing this forward at all?

@the-moliver is this still a welcome contribution? Also, I noticed that the CONTRIBUTING.md guide doesn't explicitly state that code must work with the CNTK backend - is that true, or is that document out of date?

[sallamander]

Bump @oarriaga @the-moliver