Link neural network episode concept to machine learning

[zora-wuw]

Very excited to see a deep learning course being developed! Here is some feedback I have and I hope this helps to build the course better.

• 1. Explain the reason for the activation function. e.g. With the initial weight product and the sum of bias, the input value is essentially transformed in a linear way, a straight line like linear regression from machine learning. To capture the complex patterns in our data, we need to add more complexity to the formula so an activation function is needed. There are different activation functions that can be used and each has its pros and cons. ....(List some of them so learners know there's more to learn about how to choose them accordingly/)
• 2. Add labels, softmax function and backpropagation concept e.g. For a supervised learning model, the output layer generates a score for the input, which can be used to indicate the probability of the classification result for instance. The label data are given at the output layer and the softmax function can be used to calculate the probability from the output score, which is then used in the loss function to evaluate the model performance. Similar to how a machine learns we talked about in the previous lesson, a neural network needs to update all its weights and biases according to the loss value too. This is done through backpropagation. It essentially gives the model the output and current parameters (weight and bias) for each layer and calculates the derivative of the formula, which represents the gradient. This process traverses reversely from the output layer to the input layer and finds out which parameters to change that can reduce the loss the most, and the model will do so before the next iteration begins.

[tompollard]

Thanks @zora-wuw!

Explain the reason for the activation function.

We recently added a section on activation functions. See: https://carpentries-incubator.github.io/machine-learning-neural-python/04-create_net/index.html#activation-functions. Does this address your point or do you think further explanation is needed?

Add labels, softmax function and backpropagation concept

I agree, these should be explained. I'd also like to add more detail about the example network architecture. I think we'll end up having to split https://carpentries-incubator.github.io/machine-learning-neural-python/04-create_net/ into 2 or 3 separate sections.