MichalDanielDobrzanski / DeepLearningPython

neuralnetworksanddeeplearning.com integrated scripts for Python 3.5.2 and Theano with CUDA support
MIT License
2.79k stars 1.27k forks source link

the cost of regularization in Network2.py total_cost should not in the loop? #12

Closed llgithubll closed 6 years ago

llgithubll commented 6 years ago
def total_cost(self, data, lmbda, convert=False):
        cost = 0.0
        for x, y in data:
            a = self.feedforward(x)
            if convert: y = vectorized_result(y)
            cost += self.cost.fn(a, y)/len(data)
            cost += 0.5*(lmbda/len(data))*sum(np.linalg.norm(w)**2 for w in self.weights) # '**' - to the power of.
        return cost

the cost of regularization should not in the loop?

MichalDanielDobrzanski commented 6 years ago

https://github.com/mnielsen/neural-networks-and-deep-learning/blob/86b38f5ba4d402b60167a82d638616f5e16f2f51/src/network2.py

http://neuralnetworksanddeeplearning.com/chap3.html#overfitting_and_regularization - please take a look at equation 85

Yes, it should not be in the loop, as you calculate total cost for all vectors (x and y respectively).