uber-research / sbnet

Sparse Blocks Networks
Other
435 stars 91 forks source link

sparse_conv_lib doesn't do sparse_conv2d_custom correctly #23

Closed dhingratul closed 6 years ago

dhingratul commented 6 years ago

`import sys import numpy as np import tensorflow as tf sys.path.insert(0, 'sbnet/sbnet_tensorflow/benchmark') from sparse_conv_lib import convert_mask_to_indices_custom, sparse_conv2d_custom, \ calc_block_params, sparse_conv2d, convert_mask_to_indices

size = [1, 704, 800, 6] grid = (np.random.rand(*size) > 0.95).astype(np.float32)

block_params = calc_block_params(in_size=size, bsize=[1, 3, 3, 1], ksize=[3, 3, 1, 1], strides=[1, 1, 1, 1], padding='SAME') with tf.Session() as sess:

x = tf.placeholder(tf.float32, size)
mask = x

w = tf.constant(np.ones(shape=[3, 3, 6, 64]), dtype=tf.float32)

indices = convert_mask_to_indices_custom(mask, block_params, tol=0.1)

y_dense = tf.nn.conv2d(x, w, strides=[1, 1, 1, 1], padding='SAME')
y_sparse_custom = sparse_conv2d_custom(x, w,
                                       indices,
                                       block_params,
                                       [1, 1, 1, 1], transpose=True)
# import pdb;pdb.set_trace();
print("Dense:")
a = sess.run([y_dense], feed_dict={x: grid})
print(np.array(a).shape)

print("Sparse (custom):")
b = sess.run([y_sparse_custom], feed_dict={x: grid})
print(np.array(b).shape)

`

Outputs: Dense: (1, 1, 704, 800, 64) Sparse (custom): (1, 1, 704, 800, 6)

andrei-pokrovsky commented 6 years ago

Looks like with transpose=True the filter is interpreted as RSKC instead of RSCK by tf.nn.conv2d with data_format='NCHW'. Can't remember now if it's always been the case or if the tf spec changed.

""" if transpose: q = tf.nn.conv2d(p, w, strides, 'VALID', data_format='NCHW', use_cudnn_on_gpu=True) else: q = tf.nn.conv2d(p, w, strides, 'VALID', use_cudnn_on_gpu=True) """