getting ValueError: Error when checking input: expected dense_33_input to have shape (2026,) but got array with shape (20,)

[sauravanand542]

I used this code to apply genetic algorithm on other data. I have changed the code accordingly train.txt main.txt network.txt optimizer.txt I have made changes in the 'train' and 'main' files

instead of get_mnist()/get_cifar10(), I have created my own function get_bank() which goes like this: `""" Utility used by the Network class to actually train.

Based on: https://github.com/fchollet/keras/blob/master/examples/mnist_mlp.py

""" from keras.datasets import mnist, cifar10 from keras.models import Sequential from keras.layers import Dense, Dropout from keras.utils.np_utils import to_categorical from keras.callbacks import EarlyStopping import pandas as pd

Helper: Early stopping.

early_stopper = EarlyStopping(patience=5)

def get_bank(): """Retrieve the CIFAR dataset and process the data."""

set defaults

nb_classes = 2
batch_size = 100
input_shape = (2026,)

# Get the data.
data = pd.read_csv('clean.csv')
array = data.values
x_train = array[0:2026,:20]
x_test = array[2027:4053,:20]
y_train = array[2027:4053,21]
y_test = array[2027:4053,21]

#x_train = x_train.astype('float32')
#x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

# convert class vectors to binary class matrices
y_train = to_categorical(y_train, nb_classes)
y_test = to_categorical(y_test, nb_classes)

return (nb_classes, batch_size, input_shape, x_train, x_test, y_train, y_test)

def compile_model(network, nb_classes, input_shape): """Compile a sequential model.

Args:
    network (dict): the parameters of the network

Returns:
    a compiled network.

"""
# Get our network parameters.
nb_layers = network['nb_layers']
nb_neurons = network['nb_neurons']
activation = network['activation']
optimizer = network['optimizer']

model = Sequential()

# Add each layer.
for i in range(nb_layers):

    # Need input shape for first layer.
    if i == 0:
        model.add(Dense(nb_neurons, activation=activation, input_shape=input_shape))
    else:
        model.add(Dense(nb_neurons, activation=activation))

    model.add(Dropout(0.2))  # hard-coded dropout

# Output layer.
model.add(Dense(nb_classes, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer=optimizer,
              metrics=['accuracy'])

return model

def train_and_score(network, dataset): """Train the model, return test loss.

Args:
    network (dict): the parameters of the network
    dataset (str): Dataset to use for training/evaluating

"""

nb_classes, batch_size, input_shape, x_train, x_test, y_train, y_test = get_bank()

model1 = compile_model(network, nb_classes, input_shape)

model1.fit(  x_train, y_train, 
    batch_size=batch_size,
    epochs=10000, verbose=0, 
    validation_data=(x_test, y_test), 
    callbacks=[early_stopper] )

score = model1.evaluate(x_test, y_test, verbose=0)

return score[1]  # 1 is accuracy. 0 is loss.

`

`ValueError Traceback (most recent call last)

in () 112 113 if __name__ == '__main__': --> 114 main() in main() 109 (generations, population)) 110 --> 111 generate(generations, population, nn_param_choices, dataset) 112 113 if __name__ == '__main__': in generate(generations, population, nn_param_choices, dataset) 61 62 # Train and get accuracy for networks. ---> 63 train_networks(networks, dataset) 64 65 # Get the average accuracy for this generation. in train_networks(networks, dataset) 22 pbar = tqdm(total=len(networks)) 23 for network in networks: ---> 24 network.train(dataset) 25 pbar.update(1) 26 pbar.close() ~\network.py in train(self, dataset) 46 """ 47 if self.accuracy == 0.: ---> 48 self.accuracy = train_and_score(self.network, dataset) 49 50 def print_network(self): ~\train.py in train_and_score(network, dataset) 99 validation_data=(x_test, y_test), 100 callbacks=[early_stopper] ) --> 101 102 score = model1.evaluate(x_test, y_test, verbose=0) 103 ~\Anaconda3\lib\site-packages\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs) 1152 sample_weight=sample_weight, 1153 class_weight=class_weight, -> 1154 batch_size=batch_size) 1155 1156 # Prepare validation data. ~\Anaconda3\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size) 577 feed_input_shapes, 578 check_batch_axis=False, # Don't enforce the batch size. --> 579 exception_prefix='input') 580 581 if y is not None: ~\Anaconda3\lib\site-packages\keras\engine\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix) 143 ': expected ' + names[i] + ' to have shape ' + 144 str(shape) + ' but got array with shape ' + --> 145 str(data_shape)) 146 return data 147 ValueError: Error when checking input: expected dense_33_input to have shape (2026,) but got array with shape (20,)`

[xWuWux]

The error you're encountering is related to the input shape mismatch between the expected input shape of the neural network model and the shape of the input data. Here's a step-by-step breakdown of the issue and suggestions for improvement:

Issue: The error message indicates that the neural network model expects an input shape of (2026,), but the actual input data has a shape of (20,).

Analysis: The input shape is determined by the input_shape parameter when creating the first layer of the neural network. In your case, the first layer is defined in the compile_model function.

The get_bank function returns (2026,) as the input_shape, which seems to be the expected input shape for the neural network.

However, during training in the train_and_score function, it appears that the actual input data (x_trainand x_test) has a shape of (20,).

Possible Solutions: Check Data Dimensions:

Ensure that the input data loaded from the CSV file (clean.csv) has the correct dimensions. It seems that the data might be loaded incorrectly, resulting in a shape mismatch. Review Data Loading:

Review the data loading process in the get_bank function. Confirm that the features (x_train and x_test) have the correct shape. You might need to reshape or preprocess the data appropriately.

x_train = array[0:2026, :20] Check Feature Dimension:

Confirm that the number of features matches the expected input shape. If the neural network expects (2026,) as input, each training example should have 2026 features. Debugging:

Print the shapes of x_train and x_test before training to understand their dimensions: print("Shape of x_train:", x_train.shape) print("Shape of x_test:", x_test.shape) Review Model Architecture:

Double-check the architecture of your neural network model in the compile_model function. Ensure that the first layer is correctly configured with the expected input shape. model.add(Dense(nb_neurons, activation=activation, input_shape=input_shape)) Reshape Data if Necessary:

If the data has the correct information but is not shaped appropriately, you might need to reshape it. For example: x_train = x_train.reshape((num_samples, 2026)) Check CSV Data: Inspect the contents of your CSV file to ensure that the data is formatted correctly.

Additional Suggestions:

Validation Set: Consider using a separate validation set during training to monitor model performance and prevent overfitting.

Normalization: Ensure that data normalization is applied consistently during training and testing.

Callback Monitoring: Monitor the model during training using Keras callbacks to save the best model or perform early stopping. By addressing these points, you should be able to identify and resolve the input shape mismatch issue.