search

dyigitpolat / mimarsinan

a pipeline for modeling in-memory architectures for spiking neural networks
MIT License
6 stars 0 forks source link

implement adaptive noisy training #32

Closed dyigitpolat closed 1 year ago

dyigitpolat commented 1 year ago

training with noisy activations resulted in much better conversion performance, especially with deep neural networks. however, training with too much noise also affects the pre-training performance and may prevent the network from converging to good pre-training accuracy.

like weight quantization and activation quantization adaptations, noise rate may also be adapted gradually. this can retain the pre-training accuracy while adapting the model for high noise resistance.

an additional noise adaptation step after pre-training will be a good solution

dyigitpolat commented 1 year ago

solved with: https://github.com/dyigitpolat/mimarsinan/pull/35