search

jinzishuai / learn2deeplearn

A repository of codes in learning deep learning
GNU General Public License v3.0
13 stars 1 forks source link

Question: How to calculate the derivative of a max/average function that is to be used in the back prop of the pooling? #28

Closed jinzishuai closed 6 years ago

jinzishuai commented 6 years ago

In Quiz #27

jinzishuai commented 6 years ago

ref: from the homework https://hub.coursera-notebooks.org/user/nmxunfrndctliueoiujvma/notebooks/week1/Convolution%20model%20-%20Step%20by%20Step%20-%20v1.ipynb or https://github.com/jinzishuai/learn2deeplearn/blob/master/deeplearning.ai/C4.CNN/week1_foundations/hw/Convolution%2Bmodel%2B-%2BStep%2Bby%2BStep%2B-%2Bv1.ipynb

image

Why do we keep track of the position of the max? It's because this is the input value that ultimately influenced the output, and therefore the cost. Backprop is computing gradients with respect to the cost, so anything that influences the ultimate cost should have a non-zero gradient. So, backprop will "propagate" the gradient back to this particular input value that had influenced the cost.