Open twhughes opened 5 years ago
This introduces automatic computation of the backpropagation through activation functions.
Changes:
There is only one Nonlinearity class. Real, complex, and non-holomorphic activations handled exactly the same way.
Nonlinearity
You only need to specify the forward_pass() method of each nonlinearity. Autograd will take care of the backward pass for you.
forward_pass()
This introduces automatic computation of the backpropagation through activation functions.
Changes:
There is only one
Nonlinearityclass. Real, complex, and non-holomorphic activations handled exactly the same way.You only need to specify the
forward_pass()method of each nonlinearity. Autograd will take care of the backward pass for you.