Is this the correct way to implement a hopfield network?

[raimannma]

https://github.com/liquidcarrot/carrot/blob/bf9ae22199c77911f8e6da474e04be943f92296f/src/architecture/architect.js#L466-L498

If this is correct than a hopfield network is just a simple perceptron but with STEP activation function.

I found this pics of hopfield networks:

They seem like there is more ongoing then just a dense layer with step activation, or am I wrong?

Also Wikipedia says: "A Hopfield network is a form of recurrent artificial neural network" (Wikipedia). But there are only feedforward components.

[christianechevarria]

You're absolutely right about it being more complex than just a dense feed-forward network. To my understanding in a hopfield network all nodes connect to all other nodes so implementation-wise this depends on whether the methods.connection.ALL_TO_ALL connection method is implemented correctly inside of Group.connect()

[raimannma]

ALL_TO_ALL means

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1

but there should be backward pointing connections right ?

Like this?

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

[christianechevarria]

but there should be backward pointing connections right ?

Like this?

input0 -> output0
input0 -> output1
input1 -> output0
input1 -> output1
output0 -->input0
output0 -->input1
output1 -->input0
output1 -->input1

Yes exactly, we could create a new connection method that does this. Also, ALL_TO_ALL is a bit misleading in this context I wonder if we should rename it

[raimannma]

Ok, consider renaming it to ALL_TO_ALL_FORWARD ?

And we need no additional connection type we can just do:

input.connect(output, methods.connection.ALL_TO_ALL);
output.connect(input, methods.connection.ALL_TO_ALL);

[christianechevarria]

That's true and much more elegant :+1: renaming sounds good also

[christianechevarria]

We should also have some basic structural tests for architecture to avoid things like this slipping through