At present, when the bias term is requested from the layers, an additional column in the rpu_tile is created and the biases are updated during the analog backward pass, which add noise and non-idealities typical of the analog devices. In some hardware design it may be desirable to compute the bias in digital, as it would improve the overall network accuracy.
Proposed solution
Now the bias can be either compute in the analog rpu_tile or left as pytorchnn.Parameter. This behavior can be controlled by setting the digital_bias parameter of the layers to True.
maljoras
commented
3 years ago
Description and motivation
At present, when the bias term is requested from the layers, an additional column in the
rpu_tileis created and the biases are updated during the analog backward pass, which add noise and non-idealities typical of the analog devices. In some hardware design it may be desirable to compute the bias in digital, as it would improve the overall network accuracy.Proposed solution
Now the bias can be either compute in the analog
rpu_tileor left aspytorchnn.Parameter. This behavior can be controlled by setting thedigital_biasparameter of the layers toTrue.