Error happened while building parsed model

[FlyGinger]

I am training a AlexNet on CIFAR10 and converting it to SNN.

Here is the code for AlexNet training.

from keras.datasets import cifar10

# get the training and test data
(input_train, output_train), (input_test, output_test) = cifar10.load_data()

# create model
model = keras.models.Sequential([
    keras.layers.Resizing(227, 227),
    keras.layers.Conv2D(filters=96, kernel_size=(11, 11), strides=(4, 4), activation='relu', input_shape=(227, 227, 3)),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3, 3), strides=(2, 2)),
    keras.layers.Conv2D(filters=256, kernel_size=(5, 5), strides=(1, 1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3, 3), strides=(2, 2)),
    keras.layers.Conv2D(filters=384, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.Conv2D(filters=384, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.Conv2D(filters=256, kernel_size=(3, 3), strides=(1, 1), activation='relu', padding="same"),
    keras.layers.BatchNormalization(),
    keras.layers.MaxPool2D(pool_size=(3, 3), strides=(2, 2)),
    keras.layers.Flatten(),
    keras.layers.Dense(4096, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(4096, activation='relu'),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(10, activation='softmax')
])

# compile the model
model.compile(
    loss='categorical_crossentropy',
    optimizer=keras.optimizers.SGD(learning_rate=0.001),
    metrics=['accuracy'])

# train the model
history = model.fit(input_train / 255, to_categorical(output_train), epochs=1, batch_size=32)
keras.models.save_model(model, 'AlexNet.h5')

The AlexNet can be trained correctly, so I want to convert it to SNN. Here is the config of snntoolbox.

[paths]
path_wd = /* omitted */
dataset_path = %(path_wd)s/../CIFAR10
filename_ann = AlexNet

[input]
model_lib = keras
dataset_format = npz
poisson_input = True

[tools]
evaluate_ann = False
convert = True
normalize = True

[conversion]
max2avg_pool = True

[simulation]
simulator = INI
duration = 100
num_to_test = 1000
batch_size = 100
keras_backend = tensorflow
top_k = 5

[output]
log_vars = {'spiketrains_n_b_l_t'}

Then I got an error.

Parsing input model...
Skipping layer Resizing.
Parsing layer Conv2D.
Using activation relu.
Absorbing batch-normalization parameters into parameters of previous Conv2D.
Using BatchNorm axis 3.
Skipping layer BatchNormalization.
Parsing layer MaxPooling2D.
Replacing max by average pooling.
Parsing layer Conv2D.
Using activation relu.
Absorbing batch-normalization parameters into parameters of previous Conv2D.
Using BatchNorm axis 3.
Skipping layer BatchNormalization.
Parsing layer MaxPooling2D.
Replacing max by average pooling.
Parsing layer Conv2D.
Using activation relu.
Absorbing batch-normalization parameters into parameters of previous Conv2D.
Using BatchNorm axis 3.
Skipping layer BatchNormalization.
Parsing layer Conv2D.
Using activation relu.
Absorbing batch-normalization parameters into parameters of previous Conv2D.
Using BatchNorm axis 3.
Skipping layer BatchNormalization.
Parsing layer Conv2D.
Using activation relu.
Absorbing batch-normalization parameters into parameters of previous Conv2D.
Using BatchNorm axis 3.
Skipping layer BatchNormalization.
Parsing layer MaxPooling2D.
Replacing max by average pooling.
Parsing layer Flatten.
Parsing layer Dense.
Using activation relu.
Skipping layer Dropout.
Parsing layer Dense.
Using activation relu.
Skipping layer Dropout.

Building parsed model...

Traceback (most recent call last):
  File "/.../miniconda/bin/snntoolbox", line 8, in <module>
    sys.exit(main())
  File "/.../miniconda/lib/python3.9/site-packages/snntoolbox/bin/run.py", line 49, in main
    run_pipeline(config)
  File "/.../miniconda/lib/python3.9/site-packages/snntoolbox/bin/utils.py", line 89, in run_pipeline
    parsed_model = model_parser.build_parsed_model()
  File "/.../miniconda/lib/python3.9/site-packages/snntoolbox/parsing/utils.py", line 817, in build_parsed_model
    parsed_layers[layer['name']] = parsed_layer(**layer)(inbound)
  File "/.../miniconda/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/.../miniconda/lib/python3.9/site-packages/tensorflow/python/framework/ops.py", line 1969, in _create_c_op
    raise ValueError(e.message)
ValueError: Exception encountered when calling layer "03MaxPooling2D_13x13x256" (type AveragePooling2D).

Negative dimension size caused by subtracting 3 from 2 for '{{node 03MaxPooling2D_13x13x256/AvgPool}} = AvgPool[T=DT_FLOAT, data_format="NHWC", ksize=[1, 3, 3, 1], padding="VALID", strides=[1, 2, 2, 1]](Placeholder)' with input shapes: [100,2,2,256].

Call arguments received by layer "03MaxPooling2D_13x13x256" (type AveragePooling2D):
  • inputs=tf.Tensor(shape=(100, 2, 2, 256), dtype=float32)

Why it occur? :cry:

[FlyGinger]

By the way, dataset is generated by the following python code.

from keras.datasets import cifar10

(input_train, output_train), (input_test, output_test) = cifar10.load_data()

input_train = np.expand_dims(input_train, -1)
# output_train = to_categorical(output_train, 10)
input_test = np.expand_dims(input_test, -1)
output_test = to_categorical(output_test, 10)

np.savez_compressed('x_test', input_test)
np.savez_compressed('y_test', output_test)
np.savez_compressed('x_norm', input_train[::10])

And "image_data_format": "channels_last" in Keras.json.

I am afraid that I don't know whether these matter in this problem.

[rbodo]

It seems the last pooling layer has only size 2x2 without padding, but you apply a 3x3 kernel on it. Hence the "negative dimension size" error. It is caused by the Resize layer, which is currently not implemented in the toolbox (notice the line "Skipping layer Resizing"). You can either add this layer type to the toolbox, or resize the dataset before feeding it into the network.

By the way, I notice that you are using BatchNormalization layers after an activation function. That is not possible in a converted SNN (see here).

Lastly, unless you have some reason not to, I'd recommend training the ANN with average pooling directly if you already know that you are going to use them in the SNN.