search

zishansami102 / CNN-from-Scratch

A scratch implementation of Convolutional Neural Network in Python using only numpy and validated over CIFAR-10 & MNIST Dataset
http://cnndigits.pythonanywhere.com/
GNU General Public License v3.0
271 stars 79 forks source link

Update convnet.py #8

Closed AdityaSoni19031997 closed 6 years ago

AdityaSoni19031997 commented 6 years ago

@zishansami102 added lecun_normal which uses gaussian distribution to initialise weights

AdityaSoni19031997 commented 6 years ago

capture

zishansami102 commented 6 years ago

@AdityaSoni19031997 As you can see that both of your PR has bugs. I cannot merge them. Just because training has started does not guarantee that the code is bug free. Take your time until it starts to converge. Around 20% of the 1st epoch, it should have a decent accuracy. & Be patient while writing code. ;)

AdityaSoni19031997 commented 6 years ago

https://github.com/chainer/chainer/blob/v3.1.0/chainer/initializers/normal.py#L39

AdityaSoni19031997 commented 6 years ago

i had multiple commits to fix the NUM_Filter one

Scale can be anything , if someone is happy with scale as 1 it can be modified

Done the modification

zishansami102 commented 6 years ago

Did you make it converge? Run the training, and wait to atleast 20-25% of the 1st epoch and paste the screenshot here. It should converge.

zishansami102 commented 6 years ago

For stddev. confusion I don't know. But what i have read & what the code in the above link is showing is the same. Maybe that's a little twisted model[your reference], i don't have any idea about that.

AdityaSoni19031997 commented 6 years ago

capture The accuracy is increasing and loss is going down so that's a sufficient indication of your code running like a charm...

AdityaSoni19031997 commented 6 years ago

@zishansami102

Because i had seen that Keras does it that way..

zishansami102 commented 6 years ago

Keras is different than your earlier code. In Keras : stddev = np.sqrt(scale) where scale /= max(1. , fan_in)

AdityaSoni19031997 commented 6 years ago

We should change the theta also?