Issue with feeding data

[RogerSinghChugh]

My problem: I have a program which implements your capsule network to classify images. The images are divided into three categories so it is a three way classification task. I use data generators to feed data to the capsule net model but it throws up the error given below. I have been stuck with this error for a long time, I have found some solutions to these types of problems but they all involve in using the fit() method belonging to the Sequential model api of keras. My understanding of the usage of fit method tells me that I would have to pass my data in some format as to satisfy the arguments of x and y of the fit method. I tried making a list of all the images I had by reading them using keras.preprocessing.image.load_img() into a list but I quickly found out that I won't be able to accomplish the task this way as I ran into memory errors while making the list. If someone could help me with this, I would be extremely grateful.

My data set: Two folders, one for training images and one for testing images. Further both of them are divided into three sub-folders each, namely Type 1, Type 2 and Type 3.

The error I'm getting :

ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[[[0.36862746, 0.25882354, 0.35686275], [0.37254903, 0.24705884, 0.3372549 ], [0.28235295, 0.16862746, 0.25490198], ..., [0.87843144, 0.7803922 , 0.9333334 ...

Here is the partial code that I modified:

def CapsNet(input_shape, n_class, routings):
    x = layers.Input(shape=input_shape)

    conv1 = layers.Conv2D(filters=256, kernel_size=9, strides=1, padding='valid', activation='relu', name='conv1')(x)

    # Layer 2: Conv2D layer with `squash` activation, then reshape to [None, num_capsule, dim_capsule]
    primarycaps = PrimaryCap(conv1, dim_capsule=8, n_channels=32, kernel_size=9, strides=2, padding='valid')

    # Layer 3: Capsule layer. Routing algorithm works here.
    digitcaps = CapsuleLayer(num_capsule=n_class, dim_capsule=16, routings=routings,
                             name='digitcaps')(primarycaps)

    # Layer 4: This is an auxiliary layer to replace each capsule with its length. Just to match the true label's shape.
    out_caps = Length(name='capsnet')(digitcaps)

    # Decoder network.
    y = layers.Input(shape=(n_class,))
    masked_by_y = Mask()([digitcaps, y])  # The true label is used to mask the output of capsule layer. For training
    masked = Mask()(digitcaps)  # Mask using the capsule with maximal length. For prediction

    # Shared Decoder model in training and prediction
    decoder = models.Sequential(name='decoder')
    decoder.add(layers.Dense(512, activation='relu', input_dim=16 * n_class))
    decoder.add(layers.Dense(1024, activation='relu'))
    decoder.add(layers.Dense(np.prod(input_shape), activation='sigmoid'))
    decoder.add(layers.Reshape(target_shape=input_shape, name='out_recon'))

    # Models for training and evaluation (prediction)
    train_model = models.Model([x, y], [out_caps, decoder(masked_by_y)])
    eval_model = models.Model(x, [out_caps, decoder(masked)])

    # manipulate model
    noise = layers.Input(shape=(n_class, 16))
    noised_digitcaps = layers.Add()([digitcaps, noise])
    masked_noised_y = Mask()([noised_digitcaps, y])
    manipulate_model = models.Model([x, y, noise], decoder(masked_noised_y))
    return train_model, eval_model, manipulate_model

def margin_loss(y_true, y_pred):
    L = y_true * K.square(K.maximum(0., 0.9 - y_pred)) + \
        0.5 * (1 - y_true) * K.square(K.maximum(0., y_pred - 0.1))
    return K.mean(K.sum(L, 1))

train_data_directory = "*restOfThePath*/train_roi_0.15"
test_data_dir = "*restOfThePath*/test_roi_0.15"

train_datagen = ImageDataGenerator(rescale=1. / 255)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_model, eval_model, manipulate_model = CapsNet(input_shape=(32, 32, 3), n_class=3, routings=3)
# compile the model
train_model.compile(optimizer=optimizers.Adam(lr=0.001), loss=[margin_loss, 'mse'],  # loss_weights=[1., 0.392]
                    metrics={'capsnet': 'accuracy'})
train_model.summary()

train_generator = train_datagen.flow_from_directory(train_data_directory, target_size=(32, 32), batch_size=32,
                                                    class_mode='categorical', shuffle=True)
test_generator = test_datagen.flow_from_directory(test_data_dir, target_size=(32, 32), batch_size=32,
                                                  class_mode='categorical', shuffle=True)

train_model.fit_generator(train_generator, validation_data=test_generator,
                          validation_steps=15,  # validation data size//batch size, placeholder values given
                          epochs=100,  # placeholder values given,
                          steps_per_epoch=45,  # train data size // batch size, placeholder values given
                          verbose=1)

[gunjannaik]

The input should be like [[x_train,y_train],[y_train,x_train]]

[TolerantChief]

Hi @RogerSinghChugh , did you find a solution to your problem? I'm stuck with the same error.