Bugs in SpKeras.2

[mervess]

Hello @Dengyu-Wu, I've had the chance to try SpKeras.2 recently and have come across the below bugs.

1. This one is fixed easily by commenting out the unseen file imports in the project.

   spkeras/spkeras/layers/__init__.py", line 2, in <module>
   from .MaxNorm_test import MaxNorm
   ModuleNotFoundError: No module named 'networks.snn.spkeras.spkeras.layers.MaxNorm_test'

2. I -tried to- fix this one via adding a control statement if num+1 < len(lmax).

   spkeras/spkeras/models.py", line 100, in convert
   _bias = kappa*_weights[1]/lmax[num+1]
   IndexError: list index out of range

3. I -tried to- fix this one via adding a control statement as well if num < len(l).

   spkeras/spkeras/models.py", line 104, in convert
   _weights[0] = kappa*_weights[0]/l[num]
   IndexError: list index out of range

   However, in the end another error gets thrown which I don't know how to get into. I tried running two different CNN models, one with BN layers, one without. Below error is thrown after trying the no-BN CNN:

   spkeras/spkeras/models.py", line 179, in convert
   layer.set_weights(weights[m])
   File "/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1783, in set_weights
   'shape %s' % (ref_shape, weight.shape))
   ValueError: Layer weight shape (65536, 1024) not compatible with provided weight shape (4096, 1024)

   and this error belongs to the CNN with BN layers:

ValueError: Dimensions must be equal, but are 10 and 84 for '{{node spike_activation_6/Add}} = Add[T=DT_FLOAT](Placeholder, spike_activation_6/Mul)' with input shapes: [?,10], [84].

Any fixes?

[Dengyu-Wu]

Hi @mervess , thanks for pointing out the bugs.

1. fixed.
2. & 3. It seems that they can be fixed from CNN model. There is no need to change the code inside "models.py". Can I have a look your CNN model, especially the lines before/after Add Layer? Or you can follow the ResNet Example (I just updated) and see if it helps.

[mervess]

Edit: Is it the missing add function with which you updated the SpKeras for functional APIs?

I've checked both of the examples and can't really pinpoint the reason. This is the no-BN model that I ran:

inputs = Input((32, 32, 1))
x = Conv2D(32, (5, 5), padding="same")(inputs)
x = Activation("relu")(x)
x = MaxPooling2D(padding="same")(x)
x = Conv2D(64, (5, 5), padding="same")(x)
x = Activation("relu")(x)
x = MaxPooling2D(padding="same")(x)
x = Flatten()(x)
x = Dense(1024)(x)
x = Activation("relu")(x)
x = Dense(num_classes)(x)
outputs = Activation("softmax")(x)

[Dengyu-Wu]

Edit: Is it the missing add function with which you updated the SpKeras for functional APIs? My mistake, I thought you are using the Add Layer for residual connection. There is no need to add Add in your code.

Can you change the MaxPooling2D into AveragePooling2D and try again?

Conversion of MaxPooling Layer is not included in SpKeras at this moment. It would cause a mismatch between your CNN and the resulting SNN.

[mervess]

The other model of mine includes AvgPool instead of MaxPool and BN layers onto the above model, but it crashes as well. I've tried them on my fork, they work, the MaxPool one as well; yet their accuracy levels drop around 20-30%.

1. Just to be sure, did SpKeras never support MaxPool, or is it exclusive to version 1 or 2?
2. Both Sequential and Functional models are supported in both versions, right?

[Dengyu-Wu]

Hi @mervess , I ran your model and found that the bugs only appear when excluding the bias in CNN model. It took me a bit time to fix it.

Can you have a try on the new code and see if it works?

Update list

• Removed wrong import codes
• Fixed conversion of no-bias CNN
• Fixed weight quantisation

1. Just to be sure, did SpKeras never support MaxPool, or is it exclusive to version 1 or 2? SpKeras only can convert average pool layer, just followed most paper.

2. Both Sequential and Functional models are supported in both versions, right? No, v1 only works with sequential models, but v2 supports both. The main difference is that v2 can convert Add Layer. It is prepared for the conversion of ResNet, or maybe Recurrent Neural Network (RNN) and DenseNet, so that SpKeras can support more architectures.

[mervess]

Thanks for your time @Dengyu-Wu. I've tried the new code, unfortunately it still gets stuck at the below point.

spkeras/spkeras/models.py", line 101, in convert
    _bias = kappa*_weights[1]/lmax[num+1]
IndexError: list index out of range

[Dengyu-Wu]

Hi @mervess , can you double check the architecture of your model, by running the following code?

for layer in your_mdl.layers:
    print(layer.name)

I have run the model you sent to me and it works well. There is an Activation Layer after each Dense and Convolutional Layer, like this,

input_11
conv2d_18
**activation_31**
average_pooling2d_12
conv2d_19
**activation_32**
average_pooling2d_13
flatten_8
dense_15
**activation_33**
dense_16
**activation_34**  

[mervess]

Layer (type)                 Output Shape              Param #   
=================================================================
input (InputLayer)           [(None, 28, 28, 1)]       0         
_________________________________________________________________
C1 (Conv2D)                  (None, 28, 28, 6)         156       
_________________________________________________________________
batch_normalization (BatchNo (None, 28, 28, 6)         24        
_________________________________________________________________
activation (Activation)      (None, 28, 28, 6)         0         
_________________________________________________________________
S2 (AveragePooling2D)        (None, 14, 14, 6)         0         
_________________________________________________________________
C3 (Conv2D)                  (None, 10, 10, 16)        2416      
_________________________________________________________________
batch_normalization_1 (Batch (None, 10, 10, 16)        64        
_________________________________________________________________
activation_1 (Activation)    (None, 10, 10, 16)        0         
_________________________________________________________________
S4 (AveragePooling2D)        (None, 5, 5, 16)          0         
_________________________________________________________________
flatten (Flatten)            (None, 400)               0         
_________________________________________________________________
F5 (Dense)                   (None, 120)               48120     
_________________________________________________________________
batch_normalization_2 (Batch (None, 120)               480       
_________________________________________________________________
activation_2 (Activation)    (None, 120)               0         
_________________________________________________________________
F6 (Dense)                   (None, 84)                10164     
_________________________________________________________________
batch_normalization_3 (Batch (None, 84)                336       
_________________________________________________________________
activation_3 (Activation)    (None, 84)                0         
_________________________________________________________________
OUTPUT_1 (Dense)             (None, 10)                850       
_________________________________________________________________
output (Activation)          (None, 10)                0         
=================================================================
Total params: 62,610
Trainable params: 62,158
Non-trainable params: 452

This is the model. I share the summary as the layer names are custom in this model.

[Dengyu-Wu]

Fixed by commenting "if condition" in line 222, models.py

[mervess]

Thanks, great!