How to initialize DeConv2dLayer weights to bilinear upsampling?

[JoelKronander]

Is there some simple way of initializing a Deconv2D layer with Bilinear weights?

I could write my own initalizer, but seems like a common thing to do. Would it make sense to add it as a library feature?

PS Great work with TensorLayer! :)

[zsdonghao]

@JoelKronander sorry of the late reply. I am not quite understand what bilinear weights stand for, do you mean given a matrix [[1, 4], [1, 4]], you can upsample to [[1, 2, 3, 4], [1, 2, 3, 4]]?

[JoelKronander]

Hi,

By bilinear weights I mean initializing the DeConvLayer weights so it replicates a standard bilinear image upsampling (resampling). This is used in some Semantic Segmentation papers, such as FCN e.g.

Below is some code to do that by creating bilinear weights in numpy and then passing them to a constant_initializer, then passed as the W_init parameter when creating the DeConv2dLayer. Not sure if/where it would make sense to add/contribute something like this to tensorlayer, custom initializers somewhere?

import tensorflow as tf
import tensorlayer as tl
import numpy as np
import skimage
from scipy.misc import imread, imresize, imsave

sess = tf.InteractiveSession()

imsize = 100
scale = 2
filter_size = (2 * scale - scale % 2)
num_channels = 3

#Create bilinear weights in numpy array
bilinear_kernel = np.zeros([filter_size, filter_size], dtype=np.float32)
scale_factor = (filter_size + 1) // 2
if filter_size % 2 == 1:
    center = scale_factor - 1
else:
    center = scale_factor - 0.5
for x in range(filter_size):
    for y in range(filter_size):
        bilinear_kernel[x,y] = (1 - abs(x - center) / scale_factor) * \
                               (1 - abs(y - center) / scale_factor)
weights = np.zeros((filter_size, filter_size, num_channels, num_channels))
for i in range(num_channels):
    weights[:, :, i, i] = bilinear_kernel

#assign numpy array to constant_initalizer and pass to get_variable
bilinear_init = tf.constant_initializer(value=weights, dtype=tf.float32)

x = tf.placeholder(tf.float32, [1, imsize, imsize, num_channels])

network = tl.layers.InputLayer(x, name='input_layer')
network = tl.layers.DeConv2dLayer(network,
                            shape = [filter_size, filter_size, num_channels, num_channels],
                            output_shape = [1, imsize*scale, imsize*scale, num_channels],
                            strides=[1, scale, scale, 1],
                            W_init=bilinear_init,
                            padding='SAME',
                            act=tf.identity, name='g/h1/decon2d')
y=network.outputs
sess.run(tf.global_variables_initializer())

#Random test image
img = np.random.random_sample((imsize,imsize,num_channels))

deconv_result = sess.run(y, feed_dict={x: [img]})
#skimag
upsampled_skimage = skimage.transform.rescale(img,
                                     scale,
                                     mode='constant', #zero padding as in tf conv2d_transpose
                                     cval=0,
                                     order=1, #corresponds to bilinear upsampling
                                     preserve_range=False)

print(np.allclose(upsampled_skimage,deconv_result))

imsave('skim.png',upsampled_skimage)
imsave('deconv2d.png', np.squeeze(deconv_result))

[zsdonghao]

@JoelKronander oh, great you did it. for contribution, you can put it into layers.py ; when we collect a lot of special initializer , we may create init.py

[JoelKronander]

Ok, I will submit a pull request adding a function to get such an initializer in layers.py. Where would it be appropriate to submit a corresponding test case of the function? At some point I could also look into submitting a FCN segmantic segmentation example.