There are multiple layers which are specified as being supported for second order derivatives that actually do not work when trying to calculate the Hessian diagonal using backpack-for-pytorch<=1.6.0.
So far, I've run into this problem with the following layers:
There are multiple layers which are specified as being supported for second order derivatives that actually do not work when trying to calculate the Hessian diagonal using
backpack-for-pytorch<=1.6.0
.So far, I've run into this problem with the following layers:
This can be tested with a script such as the following:
I'm guessing that these require independent fixes, but think it is a good idea to collect all layers with missing support summarised here.