Problem with custom convolutional network

[asbjornlystrup]

I'm preparing for my main project by doing some testing with the CIFAR10 dataset, for guessing objects in images. I train my networks in tensorflow 1.7.1 (is this version okay?), convert them to a .pb file, then convert it to a Unity binary .nn file.

I managed to replicate similar accuracy in Unity when using a very simple network that only does a linear transformation. The imported network in Unity's console window: https://i.imgur.com/1lZ44S2.png I got an error with BarracudaWorkerFactory.Type.ComputeFast, so it uses BarracudaWorkerFactory.Type.Compute.

I then moved over to a convolutional network, which Unity imports as such: https://i.imgur.com/PApzFGu.png However, despite the accuracy in tensorflow being 60%, Unity returns 10% (as in, a completely random guess because there are only 10 categories of items to guess from). This is what the tensorflow_to_barracuda.py script prints in the console:

Converting graph.pb to graph.nn GLOBALS: 'Reshape/shape' OUT: 'output_node' DONE: wrote graph.nn file.

A bit strange, as it's not printing the input node like in the simple linear network:

Converting graph.pb to graph.nn GLOBALS: 'Reshape/shape' IN: 'input_node': [-1, 32, 32, 3] => 'Reshape' OUT: 'output_node' DONE: wrote graph.nn file.

The tensorflow code for the convolutional network is this:

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.01)
    return tf.Variable(initial)

def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)

def conv2d(x, W, stride=2):
    return tf.nn.conv2d(x, W, strides=[1, stride, stride, 1], padding='SAME')

n = 16

x = tf.placeholder(tf.float32, shape=[None,32,32,3], name="input_node") # input batch of images
y_ = tf.placeholder(tf.int64, shape=[None]) # input labels

W = weight_variable([4, 4, 3, n])
b = bias_variable([n])

hidden_1 = tf.nn.relu(conv2d(x, W) + b)

W_2 = weight_variable([4, 4, n, n * 2])
b_2 = bias_variable([n * 2])
hidden_2 = tf.nn.relu(conv2d(hidden_1, W_2) + b_2)

W_3 = weight_variable([4, 4, n * 2, n * 4])
b_3 = bias_variable([n * 4])
hidden_3 = tf.nn.relu(conv2d(hidden_2, W_3) + b_3)

W_4 = weight_variable([4 * 4 * n * 4, classes])
b_4 = bias_variable([classes])
hidden_3_flat = tf.reshape(hidden_3, (-1, 4 * 4 * n * 4))
y = tf.add(tf.matmul(hidden_3_flat, W_4), b_4, name='output_node')

There's nothing wrong with the .pb files, as I imported those back into python and ran tests returning the exact same accuracies. So the problem should lie either in the tensorflow_to_barracuda script or in Unity.

Do you know what could be wrong? Is there some tool that can convert .nn back to .pb for testing, or a way of viewing .nn graphs in Tensorboard?

Thank you in advance.

Edit: The Python version I've used is 3.6.8 64-bit Edit 2: This was with the master branch. Are there some other branches I could try? Edit 3: A lot of edits here, but I was also wondering if you support deconvolution, as in conv2d_transpose anywhere? I will need this for the main project once this CIFAR test is done. In short, the main project's goal is to use a cGAN for translating procedurally generated heightmaps into eroded mountains based on satellite images. If deconvolution is not supported, I assume TensorFlowSharp would work as an alternative, right?

[shihzy]

@mantasp

[mantasp]

@Mytino any chance you could provide sample project that demonstrates the issue? Thanks!

[asbjornlystrup]

I ended up using the v0.5.0a-branch TensorFlowSharp version and haven't looked too much back on this. I looked for the old project in my files, but the neural net files were missing in it. It would take some time to put back together, so I will close this thread as I don't have much time at the moment. I might instead come back with a new post if I run into the same problem in the future. Thank you.

[Bricktheworld]

Have you tried TensorFlowSharp with android? Did you manage to get it working and did you have an error with duplicate dlls?

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.