posts/simple_1-layer_neural_network_for_mnist_handwriting_recognition/

[utterances-bot]

Simple 1-layer neural network for MNIST handwriting recognition | Matt's Tech Blog

Simple 1-layer neural network for MNIST handwriting recognition

http://localhost:1313/posts/simple_1-layer_neural_network_for_mnist_handwriting_recognition/

[mmlind]

Test

[mmlind]

Original author: spac3hit @spac3hit
Original date: 2017-06-18T08:59:42Z

Achieved accuracy of 93.0200% with 110 nodes in the hidden layers.

[mmlind]

Original author: Matt @machinelearninginc
Original date: 2017-06-18T10:03:17Z

Wow .. that's quite a model for solving MNIST! :) How big are your layers?

[mmlind]

Original author: spac3hit @spac3hit
Original date: 2017-06-18T10:47:31Z

110 hidden layers.

[mmlind]

Original author: Matt @machinelearninginc
Original date: 2017-06-18T13:31:18Z

Yes I got that. But how many cells/nodes in each layer? And all fully connected or also using convolutional layers?

[mmlind]

Original author: spac3hit @spac3hit
Original date: 2017-06-19T06:52:43Z

All are fully connected. # of nodes in input and output layers are the same, 28 and 10 respectively. # of nodes in the hidden layer are 110. (Edited my first comment)

[mmlind]

Original author: WWondo @wwondo
Original date: 2017-07-17T08:09:53Z

Hello, Mr Matt.
I try to make image recognition using deep learning and I luckily find your blog.
Thankyou for the greatful work.
I'm now carefully examining code you provide but just to make sure for the usage.

To add FULLY_CONNECTED type layer, having 100 nodes
is it correct to make LayerDefinition like bellow?

LayerDefinition hiddenLayer3 = {
.layerType = FULLY_CONNECTED,
.activationType = SIGMOID,
.nodeMap = (Volume){.width=100},
};

I dont get what I have to put nodeMap, and filter.

Could you please give me brief usage of the code?

[mmlind]

Original author: WWondo @wwondo
Original date: 2017-07-18T04:52:59Z

I have carefully valifying the code and figure out the usage of the code.

for those who had simillar questioning as I had,
check the "netdefs.c" file that Mr Matt provided in github.

I hope I could contribute to this code sooner or later

[mmlind]

Original author: Despoina Touska @despoinatouska
Original date: 2017-07-28T02:33:29Z

Hello, very nice job Mr Matt!
I would like to ask how you manage to open the MNIST files because a have a problem with that and i have tried all the different ways in c.

[mmlind]

Original author: Matt @machinelearninginc
Original date: 2017-07-28T02:40:42Z

Checkout the code at https://github.com/mmlind/m...

[mmlind]

Original author: WWondo @wwondo
Original date: 2017-07-28T06:37:32Z

Hi, I have a question but, dont know how to contact you, so posted the issue here.

Firstly, I really appreciate your work as I'm studying neural network, and try to implement in C.
it's really helpful to study by your blog and code, thankyou.

My question is how did you construct your network to get high performance by using convolution network, like you showed on the blog (like below).

==========================================================
| Network | # of Nodes | Act.Fct | L-Rate | # Trains | Speed* | Accuracy || Compare** |
| ----------- |----------------|---------|--------|----------|--------|----------||-----------|
| 1-l FC NN | 10 output | SIGMOID | 0.0125 | 900,000 | 228s | 91.09% || 88.00% |
| 2-l FC NN | 300 hidden | SIGMOID | 0.0700 | 120,000 | 697s | 94.67% || 96.40% |
| 3-l FC NN | 500+150 hidden | SIGMOID | 0.0050 | 180,000 | 2,510s | 93.64% || 97.05% |
| 3-l ConvNet | 13x13x5, 6x6x5 | RELU | 0.0004 | 525,000 | 908s | 88.40% || 98.90% |

==========================================================

When I tried the same 13x13x5, 6x6x5 convolutional network, but keep getting 9.87%.

I cloned your code and simply run as your default main.c has 13x13x5, 6x6x5 convolutional network.

and one more thing,
when try to use relu as activation function, it also do not work, keep getting around 9% accuracy
for just simply 1-hidden layer with relu neurons, I'm keep varifing you code but dont get why.

Can you please help me with usage of your convolutional network code?

[mmlind]

Original author: Matt @machinelearninginc
Original date: 2017-07-28T06:43:03Z

Try playing with different learning rates. A 0.1 difference can cause accuracy to deteriorate drastically.

[mmlind]

Original author: Despoina Touska @despoinatouska
Original date: 2017-07-29T14:37:08Z

Very good results! Did you remember how much time did it take?

[mmlind]

Original author: spac3hit @spac3hit
Original date: 2017-07-29T16:14:47Z

I will try to run it again next week and tell you.

[mmlind]

Original author: D V
Original date: 2017-08-17T22:26:07Z

Hi, Did it take 19 sec just for 1 cycle through all 70000 images or was it 19 sec to teach the network completely ?
How many cycles did you do per 19 sec to teach the net?
Does your code involve GPU acceleration?
For my simple network on Visual Basic with 10 hidden nodes(RELU activation) and one output node it took 20 min to make 1 loop through 60 000 images and when I switched to batch update it took 13 min. Works well for 1000 images though

[mmlind]

Original author: Matt @machinelearninginc
Original date: 2017-08-19T23:46:35Z

I used online learning (updating weights after every sample image) and trained the network in one run through 60k (training and validation) images. So only one cycle which took about 10secs.

[mmlind]

Original author: Alexandra Beebe @alexandrabeebe
Original date: 2017-11-06T21:20:24Z

I think I am doing something wrong. I am using your dnn code to try and reproduce the results you were getting with 1lnn, and 3lnn. But every time I run the tests I am getting accuracy around 11%

It it not possible to reproduce the 1lnn and 3lnn in the dnn?

[mmlind]

Original author: David Demenongu @daviddemenongu
Original date: 2017-11-28T00:36:18Z

david d

[mmlind]

Original author: David Demenongu @daviddemenongu
Original date: 2017-11-28T00:38:58Z

please my project supervisor approve a topic for my final year project

Machine Learning for recognizing hand written digits (MNIST dataset) and am finding it difficult where to start and how to go about it, please help me out please

[mmlind]

Original author: Sharjeel Mansha @sharjeelmansha
Original date: 2018-03-29T16:30:46Z

Hi David you can contact me i can help you in that case
email: manshasharjeel@gmail.com

[mmlind]

Original author: Matthew Wong
Original date: 2018-05-17T22:54:44Z

Hi,

Thanks so much for the article!

I have a quick question. When I downloaded your code and ran it, I noticed that some of the weights were negative. This seems to make sense since you didn't have a floor on the value that weights can take.

However, in the article you write that "The input is binary [0,1] and the weight is scalar [0-1]. Thus, a cell’s output would be somewhere in the range of [0-794]."

Did you mean the weights are in the range [-1, 1] and the outputs are in the range [-794, 794]?

Thanks!

Cheers,
Matthew

[mmlind]

Original author: Ola
Original date: 2019-01-28T19:53:18Z

Great tutorial, thanks a lot ;)
28 * 28 = 784, just saying :)

Also, I wanted to ask if activation function is irrelevant here? Normalization of output by dividing it by 28*28 is enough?

Edit: Must be, because it works like charm ;) 82,5 % rate in half of first training. Thanks again!

Edit2: test data 85%. but i fail to save weights in file and read them again, this is legal procedure right?

Edit3: I've develop my own data set and get 95% accuracy in auto test, and 90% - 100% when testing manually.

[mmlind]

Original author: Ashley Emerson @lectpipagyp1992
Original date: 2019-02-28T08:43:56Z

Hi I wanna ċhαt with you😏
https://google.com/#btnI=ru...
My id 794531

[mmlind]

Original author: Nickie Niaki @nickieniaki
Original date: 2019-04-15T16:34:13Z

Can you simply use binary classification for each class [0, 1, 2, ..., 9] and in the end take the maximum between probabilities to determine the predicted label?
I think Softmax Regression needs to be applied to this classification task.

[mmlind]

Original author: t.k. @ga6840
Original date: 2019-09-07T14:45:59Z

One layer achieved 85% accuracy is still amazing.

[mmlind]

Original author: Cem KIRAC @cemkirac
Original date: 2019-10-23T12:05:59Z

Thank you very much

[mmlind]

Original author: legend copymate @legendcopymate
Original date: 2020-06-19T21:06:37Z

plz share this in video form

[mmlind]

Original author: legend copymate @legendcopymate
Original date: 2020-06-19T21:08:11Z

dear i found susceptibility through logistic regression by using the raster data so plz share video or mail on my email address i am vry thankful to u