Simple neural net has different output for inputs in the same batch that are identical

[happypepper]

In this working standalone code example, an nn is initialized with 2 linear layers. The inputs are then filled with 0s and passed through the NN.

As you can see, the nn outputs are identical for the first 4 inputs of the batch, but the 5th one is different from the 4th.

require 'nn'
require 'torch'

torch.setdefaulttensortype('torch.FloatTensor')
torch.manualSeed(0)

local my_nn = nn.Sequential()
my_nn:add(nn.Linear(1009, 500))
my_nn:add(nn.Linear(500, 1008))

inputs = torch.FloatTensor(5,1009):fill(0)
values = torch.FloatTensor(5,1008)

values:copy(my_nn:forward(inputs))
print(torch.all(torch.eq(values[1], values[2])))
print(torch.all(torch.eq(values[2], values[3])))
print(torch.all(torch.eq(values[3], values[4])))
print(torch.all(torch.eq(values[4], values[5])))
print(values[4][1008])
print(values[5][1008])

output:

true    
true    
true    
false   
-0.0064480220898986 
-0.0064480230212212

[tastyminerals]

You should read about comparing float numbers first. The output of network is correct.

[happypepper]

Why is the output correct? What reason is there for the 5th output in the batch to be different than the 4th output in the batch, but 1st - 4th outputs are identical?

There's nothing wrong with floating point comparison, it will just compare to see if the binary representations of each floating point number is equal. In this case, they are indeed equal for 1st - 4th outputs.

[tastyminerals]

Sorry, yeah, haven't read your code. Running on my machine.

true    
true    
true    
true    
-0.0064480295404792 
-0.0064480295404792

How did you compile it?

[happypepper]

That is intriguing, do you have an idea why this would happen?

[tastyminerals]

Well, first thing is to make sure you have compiled and installed torch according to official instructions.

[tastyminerals]

Lua5.1, gcc-4.9 linked against system gcc.

[happypepper]

I've installed with Lua 5.2, I believe it is supported as per http://torch.ch/docs/getting-started.html

running the code with th example.lua

[tastyminerals]

Since I never used Lua 5.2 with Torch I can only suggest you compile and install Torch with Lua 5.1 and check again. There is absolutely no reason to have Lua 5.2 unless you want to deal with such unexpected behaviour.

[happypepper]

I just reinstalled with Lua 5.1, but still getting the same result. I'm using mac OS. How can I check the gcc version used?

[tastyminerals]

gcc --version in terminal. You wouldn't be able to compile it with newer gcc version so whatever is installed should be correct. Hmm, strange indeed.

[happypepper]

I just ran the same code on linux with Lua 5.2 installed and it gave the correct result. Can someone else with Mac OS confirm the code also gives incorrect result on their machine?