Lower SNN accuracy compared with ANN accuracy

[jimzhou112]

Hello,

First off, thank you so much for the help you have already provided. It's been invaluable.

I am using SNN Toolbox to convert a Keras/TF ANN model to SNN to run on Intel's Loihi chip. The resulting simulation accuracy numbers for SNN are much lower than ANN. My model achieves 34.3% accuracy on ANN but the SNN accuracy is 21%. My model architecture is attached below and the input does not undergo any normalization with individual pixels values ranging from 0 to 46 (I've found that the SNN performs better without normalizing them to be between 0 and 1). Are there any techniques I can employ to close this accuracy gap?

Model architecture in TF/Keras:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (NxInputLayer)       (1, 7, 7, 1)              0         
_________________________________________________________________
0Conv2D_5x5x6 (NxConv2D)     (1, 5, 5, 6)              60        
_________________________________________________________________
1Flatten_150 (NxFlatten)     (1, 150)                  0         
_________________________________________________________________
2Dense_23 (NxDense)          (1, 23)                   3473      
=================================================================
Total params: 3,533
Trainable params: 3,533
Non-trainable params: 0
_________________________________________________________________

Thanks config.txt

[jimzhou112]

I realized that there is actually an entire research paper included with SNN Toolbox that answers the question very comprehensively. Apologies for the silly question.

[rbodo]

Not silly at all. A useful tool to inspect the accuracy drop is the correlation plots (comparing spikerates in SNN and activations in ANN). This post may also give some ideas. And since you are running on Loihi, the weight quantization may have an effect. To check that, you could quantize your model weights manually and see how the accuracy drops in the ANN (before conversion). I'd also try the builtin INIsim simulator just to check whether this model can be converted well when discarding the Loihi constraints (remember to turn on normalization in the [tools] section then).