Closed Elliot1299 closed 8 years ago
Hi @Elliot1299, You should really work with the code on this repo, rather than the old code available on StackOverflow.
The network can be instanced to a topology of your own choosing. If you construct the network using:
settings = {
"n_inputs" : 3,
"layers" : [ (3, tanh_function), (3, tanh_function),(1, sigmoid_function) ],
}
# initialize the neural network
network = NeuralNet( settings )
the network will be a 2-hidden layer neural network with a single output. In literature, this is either called a 2-hidden layer network or simply a 3-layered network. This seems to be what you are trying to achieve.
If you are interested in computing the gradients, you can call the gradient calculation routine directly:
settings = {
# Required settings
"n_inputs" : 2, # Number of network input signals
"layers" : [ (3, tanh_function), (3, tanh_function), (1, sigmoid_function) ],
}
# initialize the neural network
network = NeuralNet( settings )
# calculate the gradient for the whole network
# @param training_data This is a NumPy matrix with shape (n_samples, n_input_signals)
# @param training_targets This is a NumPy matrix with shape (n_samples, n_output_signals)
gradients = network.gradient( network.get_weights(), training_data, training_targets )
Important: Note that this routine returns all the gradiens in a single list! If you would like to have them structured by layer, swap out the following lines of code in neuralnet.py
:
for i in layer_indexes:
# Loop over the weight layers in reversed order to calculate the deltas
# calculate the weight change
dW = layers.append(np.dot( delta, add_bias(input_signals[i]) ).T.flat)
if i!= 0:
"""Do not calculate the delta unnecessarily."""
# Skip the bias weight
weight_delta = np.dot( self.weights[ i ][1:,:], delta )
# Calculate the delta for the subsequent layer
delta = weight_delta * derivatives[i-1]
#end weight adjustment loop
return np.hstack( reversed(layers) )
with:
for i in layer_indexes:
# Loop over the weight layers in reversed order to calculate the deltas
# calculate the weight change
layers.append(np.dot( delta, add_bias(input_signals[i]) ).T)
if i!= 0:
"""Do not calculate the delta unnecessarily."""
# Skip the bias weight
weight_delta = np.dot( self.weights[ i ][1:,:], delta )
# Calculate the delta for the subsequent layer
delta = weight_delta * derivatives[i-1]
#end weight adjustment loop
return reversed(layers)
If you're only interested in training a multilayered network, you should emperiment with the scg()
routine as long as you are not using too many nodes and layers,
Good morning, dear respective jorgenkg!
Thank you very much for explaining creating network structure and Gradient Calculation in detail!
Good afternoon, dear respective jorgenkg!
If is it possible may I ask following, would it be possible to set MLP with more than 2 or >2 hidden layers in your code. For example if I need to create MLP with 4 layers (1 input layer, 2 hidden layers, 1 output layer), each layer have 3 neurons, 3 inputs at input layer, 3 outputs on the output layer. In main.py I should set following: "n_inputs" : 3, # Number of network input signals "layers" : [ (3, tanh_function), (3, tanh_function),(1, sigmoid_function) ], ?
Could you please say would it be right?
Could you please say in your question at http://stackoverflow.com/questions/17049321/mlp-neural-network-calculating-the-gradient-matrices you have enclosed Code which computes Error Gradient for MLP's with n hidden layers with n neurons, may I ask you how is it possible to launch it? I have seen there 3 tabs "My Python code (derived from Chuck Anderson's snippet):", "The NPNeuralNet class", "The final implementation", what should I do in order to launch it, I should perceive "My Python code (derived from Chuck Anderson's snippet):" as main? But in "The final implementation" there is no Class declaration, only Functions definitions (def forward(), def backprop(), def errorFunction(), def errorGradient() ), maybe I should transfer them in "The NPNeuralNet class", but then I do not see calling of all this functions in "My Python code (derived from Chuck Anderson's snippet):". Could you please say how can I launch program described in your question? Currently I am studying calculations of Error gradient with appliance of Partial derivatives I have done couple of calculations manually and wanted to check it with some program but on the internet there was no any frameworks for MLP's with support of n hidden layers, except your Code which I have found on stackoverflow, as in "The NPNeuralNet class" I have seen that it is possible to set n numbers of hidden layers.
self.n_inputs = n_inputs self.n_hiddens = n_hiddens self.n_outputs = n_outputs self.n_hidden_layers = n_hidden_layers
Thank you in advance for your reply!
Sincerely