SRM & Loihi Neural Dynamics, Parameters?

[ammar-bitar]

Hi,

I was wondering if there was some way to replicate the SRM neural behaviour with the Loihi parameters?

Training a couple of SNNs using the SRM model on MNIST/F-MNIST I got to 95%+ accuracy; but trying to do the same with the Loihi model the learning happens at much much slower rate (in term of epochs).

Where, with the SRM model accuracy goes above 90% very quickly, after ~3-5 epochs (depending on coding); with the Loihi neuron-model (and the 'default' parameters, the ones you're using for NMNIST in the tutorial), it's only residing at around 50-60%.

Regarding SLAYERAuto, I'm still not 100% sure about its syntax.

I have this network:

cnnNetwork( (slayer): spikeLayer() (conv1): _convLayer(1, 16, kernel_size=(5, 5, 1), stride=(1, 1, 1), padding=(1, 1, 0), bias=False) (conv2): _convLayer(16, 32, kernel_size=(3, 3, 1), stride=(1, 1, 1), padding=(1, 1, 0), bias=False) (conv3): _convLayer(32, 64, kernel_size=(3, 3, 1), stride=(1, 1, 1), padding=(1, 1, 0), bias=False) (pool1): _poolLayer(1, 1, kernel_size=(2, 2, 1), stride=(2, 2, 1), bias=False) (pool2): _poolLayer(1, 1, kernel_size=(2, 2, 1), stride=(2, 2, 1), bias=False) (fc1): _denseLayer(64, 10, kernel_size=(8, 8, 1), stride=(1, 1, 1), bias=False) )

When trying to rewrite it with SLAYERAuto like this:

'layer' : [ {'dim': '34x34x1'}, {'dim': '16c5z'}, {'dim': '2a'}, {'dim': '32c3'}, {'dim': '2a'}, {'dim': '64c3'}, {'dim': 10} ],

I end up with this:

Network( (slayer): spikeLayer() (blocks): ModuleList( (0): convBlock( (slayer): spikeLayer() (weightOp): _convLayer(1, 16, kernel_size=(5, 5, 1), stride=(1, 1, 1), padding=(2, 2, 0), bias=False) ) (1): poolBlock( (slayer): spikeLayer() (weightOp): _poolLayer(1, 1, kernel_size=(2, 2, 1), stride=(2, 2, 1), bias=False) ) (2): convBlock( (slayer): spikeLayer() (weightOp): _convLayer(16, 32, kernel_size=(3, 3, 1), stride=(1, 1, 1), padding=(1, 1, 0), bias=False) ) (3): poolBlock( (slayer): spikeLayer() (weightOp): _poolLayer(1, 1, kernel_size=(2, 2, 1), stride=(2, 2, 1), bias=False) ) (4): convBlock( (slayer): spikeLayer() (weightOp): _convLayer(32, 64, kernel_size=(3, 3, 1), stride=(1, 1, 1), padding=(1, 1, 0), bias=False) ) (5): flattenBlock() (6): denseBlock( (slayer): spikeLayer() (weightOp): _denseLayer(5184, 10, kernel_size=(1, 1, 1), stride=(1, 1, 1), bias=False) ) ) )

(The layers are conv1 -> pool1 -> conv2 -> pool2 -> conv3 -> dense)

Especially, in the first convlayer, the padding is not the same (which might impact the learning a bit?).

Thanks!

[bamsumit]

You may start by making the time constants in similar range. The hyperparameters are going to be different because the neuron model and dynamics are different. In addition Loihi model is a fixed precision model. So your convergence is going to be slow.

On padding, just add a pad padding field in the layer description:

'layer' : [
        {'dim': '34x34x1'},
        {'dim': '16c5z', 'padding': 1},
        {'dim': '2a'},
        {'dim': '32c3'},
        {'dim': '2a'},
        {'dim': '64c3'},
        {'dim': 10}
],

[ammar-bitar]

Thanks for the reply!

Adding the 'pad' field, doesn't seem to work though, when calling the assistant to build the net using I'm still gettting: Found out looking through the source code, if I use 'padding' : 1 instead of 'pad' : 1 it works!

When you talk about the time-constants, do you mean the simulation time, sample-time, target-regions - and tau/scale Rho? If yes, those are all already set to the same!

Does it make sense to fix the Loihi threshold (vThMant) to be the same as theta, and then work with the voltage/current params?

Thanks again!

ETA:

For reference my net parameters:

netDesc = {
    'simulation' : {'Ts': 1.0, 'tSample': 300, 'nSample': 12},   #nSample: 12
    'neuron' : {
        'type'     : 'LOIHI',     
        'vThMant'  : 10, 
        'vDecay'   : 128,         
        'iDecay'   : 1024,       
        'refDelay' : 1,           
        'wgtExp'   : 0,           
        'tauRho'   : 1,          
        'scaleRho' : 1,         
    },
}

And SRM params:

netDesc = {
    'simulation' : {'Ts': 1.0, 'tSample': 300, 'nSample': 12},   #nSample: 12
    'neuron' : {
        'type'     : 'SRMALPHA',      # Neuron Model
        'theta'    : 10,
        'tauSr'    : 10.0,
        'tauRef'   : 1.0,
        'scaleRef' : 2,           # Relative to theta
        'tauRho'   : 1,           # Relative to theta 
        'scaleRho' : 1,
    }
}

[bamsumit]

Whatever is in the neuron field needs to be tuned.