search

etrommer / torch-approx

GPU-accelerated Neural Network layers using Approximate Multiplications for PyTorch
https://etrommer.de/torch-approx
MIT License
7 stars 3 forks source link

Layer Noise Mode Interface #12

Closed etrommer closed 1 year ago

etrommer commented 1 year ago

The noise mode of the layer currently adds a zero mean tensor with learnable standard deviation.

https://github.com/etrommer/torch-approx/blob/5740d50c8abfd714d86ab6286b27aa3b545cdee9/src/torchapprox/layers/approx_layer.py#L50-L60

This is quite specific to the origin of torch-approx as a backend for AGN Approx. To make it more generally useful, this feature should be kept on a separate branch and the noise implementation replaced with a more generic interface that adds Gaussian Noise of a fixed mean and standard deviation to the layer output

https://github.com/etrommer/torch-approx/blob/8284cf7953ad9bde1d3c4e4c0ce9d5c67f4ff7bd/src/torchapprox/layers/approx_layer.py#L48-L74

etrommer commented 1 year ago

Changes merged from b59d52f2e9b4c405e79561c2be082e351a4422f5

Old interface is tracked in compatibility/agnapprox