Artificial Neural Networks

[ghost]

A lot of things seem to use ANN for prediction in fields that are really far from biology and brains eg. convolutional neural networks. How are they using neural networks exactly?

[ElanHR]

Essentially Neural Networks can be viewed as a function mapping arbitrary inputs to outputs. Since the function is extremely expressible and we have methods of training them it makes them an attractive choice for classification.

ANN's are a form of stacking different classifiers. Essentially you have an input and output layer with various options of sizes, depths, and types of intermediate(hidden) layers. One reason ANN's are useful because they allow us to automatically learn a good representations for the problem in the intermediate layers (hierarchical learning).

The simplest ANN has only a single hidden layer and can be represented in the following matrix form: f(x) = G2( b2 + W2( G1( b1 + W1 x))) where G1/G2 are activation functions, W1/W2 are weight functions and b1/b2 are bias terms.

This pattern can be extended to any number of layers however the issue with deep neural networks is how to efficiently train them. Assuming nonlinear activation functions even a 2 layer neural network finding the optimal weights analytically is NP hard, therefore we most commonly optimize using gradient methods (back-prop etc). Unfortunately the deeper the network goes the more the gradient (signal) vanishes making learning exceedingly slow (there are however a number of methods to counter this - random dropout, greedy layerwise training, etc).

[ajulian3]

There are a lot of benefits from using ANN. Specifically, these layers really help localize and quantify interconnectivity to best create accurate mathematical models. The weighting, bias, and activation functions are present in ANN while not in ENN. There are issues taking into the topology into account. What are ways to take the brains shape into account when creating a mathematical model?

[mrjiaruiwang]

A drawback of neural networks however is that there is often so many layers and relationships that one cannot fully understand how it functions. There are simply too many things connected to wrap one's head around. One might prefer other classifiers such as k-nearest-neighbor over ANN because those are simpler and easier to understand.

[adjordan]

Can the algorithm be adjusted to fix layers/relationships to understandable quantities?