It's a way of calculating the gradient of a (perhaps very complex, non-linear) function by expressing the calculation of the function as a directed graph going left to right (i.e, forward), then following some simple rules to calculate the gradient from right to left (i.e., backward).
It's a way of calculating the gradient of a (perhaps very complex, non-linear) function by expressing the calculation of the function as a directed graph going left to right (i.e, forward), then following some simple rules to calculate the gradient from right to left (i.e., backward).