MLP

[martinapugliese]

add to the perceptron note

[martinapugliese]

• [ ] add leargnin in the perceptron, see https://en.wikipedia.org/wiki/Perceptron
• [ ] add network of perceptrons

[martinapugliese]

From the notebook I had on this:

Learning

The perceptron is an ingenious mechanism, but rather difficult to use for learning in a typical case. Due to the activation function being a step, a small modification to the input will produce a jump in the output, making it rather clunky to tune for a general problems. You can use it for learning things, but you have to painstakingly work on the features you inject, this video from a course by G Hinton is worth a watch. More interesting activations are indeed used these days, you can find a discussion on the activations page in this section.

However, let's not give the impression that the perceptron is useless. It still an awesome idea, and good enough for some applications. This page on Applied Go gives a very nice introduction to it and its use in a binary classification task (with Go code!).

Multilayer Perceptron and pinch of salt

A multilayer perceptron (MLP) is a term used for a network with 3 layers and a feedforward architecture. The activations used in the neurons are not necessarily Heaviside as for the perceptron neuron, and this creates a bit of confusion! Consult Wikipedia.