Problem in Error/Delta for layer 3

[guilhermefarto]

Hello Jie, how are you doing? I'm trying to build a 3-layer NN and came to your code. But, my results are different from yours. I made some tests and notice one thing that I would like to share with you. You code calculates the error for 3-layer (named del4), but del4 is the error (how much did we miss the target value). I think you need another varible to store the real delta between error for 3-layer (maybe rename del4 to err4) and a new variable 'del4' that will receive err4*self.__sigmoid_derivative(output).

So, could you please check if my following suggestion is correction?

Change this

    # calculate error
    del4 = training_set_outputs - output

    # find errors in each layer
    del3 = dot(self.synaptic_weights3, del4.T)*(self.__sigmoid_derivative(a3).T)
    del2 = dot(self.synaptic_weights2, del3)*(self.__sigmoid_derivative(a2).T)

To this

    # calculate error
    err4 = training_set_outputs - output

    del4 = err4*self.__sigmoid_derivative(output)

    # find errors in each layer
    del3 = dot(self.synaptic_weights3, del4.T)*(self.__sigmoid_derivative(a3).T)
    del2 = dot(self.synaptic_weights2, del3)*(self.__sigmoid_derivative(a2).T)

All the best! :)

[jiexunsee]

Hi Guilherme,

Apologies for the late reply. Your suggestion is right, I should have multiplied the error with the sigmoid derivative of the output before backpropagating it.

Thank so much! I shall update my code.

All the best :)

[guilhermefarto]

Hello Jie. You're welcome. No problem! :-) All the best,