In this pull request, we have made the following changes:
_jacobian_mult is renamed to _jacobian_wrt_input_mult_left_vec
We implement right multiplication for linear, conv2d, maxpool, upsampling, tanh layers. We refer to this as _jacobian_wrt_input_mult_right_vec
We implement jacobian wrt. weights for linear and conv2d layers. We refer to these functions as _jacobian_wrt_weight_mult_right_vec and _jacobian_wrt_weight_mult_right_vec
We implement the Generalized Gauss Newton (GGN) sandwich product wrt. weights and input for linear, conv2d, maxpool, upsampling, tanh layers. We refer to this as _jacobian_wrt_input_sandwich and _jacobian_wrt_weight_sandwich. We implement these GGN product for different approximations, namely diag_inp and diag_out referring to the approximations wrt. input and output respectively.
In this pull request, we have made the following changes:
_jacobian_mult
is renamed to_jacobian_wrt_input_mult_left_vec
_jacobian_wrt_input_mult_right_vec
_jacobian_wrt_weight_mult_right_vec
and_jacobian_wrt_weight_mult_right_vec
_jacobian_wrt_input_sandwich
and_jacobian_wrt_weight_sandwich
. We implement these GGN product for different approximations, namelydiag_inp
anddiag_out
referring to the approximations wrt. input and output respectively.