autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 270 forks source link

Incompatible shape on Stacked LSTM #505

Closed minhng22 closed 3 years ago

minhng22 commented 3 years ago

I got the following warning when optimizing LSTM model with Talos. Talos version '1.0.0' WARNING:tensorflow:Model was constructed with shape (None, 9, 1) for input Tensor("lstm_input:0", shape=(None, 9, 1), dtype=float32), but it was called on an input with incompatible shape (None, 6, 1).

My model is a stacked LSTM model. The code is as such:

def LSTMTrainOptimization(X_train, y_train, X_val, y_val, params):
    #Build the model
    model = Sequential()
    model.add(LSTM(
        params['NUM_NEURONS_FIRST_LAYER'],
        input_shape=(params['LOOK_BACK'], X_train.shape[2]), 
        return_sequences= True))
    model.add(Dropout(params['DROP_OUT']))
    model.add(LSTM(params['NUM_NEURONS_SECOND_LAYER'],input_shape=(params['NUM_NEURONS_FIRST_LAYER'],X_train.shape[2])))
    model.add(Dense(
        params['FORWARD_DAYS'],
        activation= params['DENSE_ACTIVATION']
    )) 
    model.compile(
        loss=params['LOSS'], 
        optimizer= params['OPTIMIZER'], 
        metrics= [tf.keras.metrics.RootMeanSquaredError()])

    history = model.fit(
        X_train,y_train,
        epochs= params['EPOCHS'],batch_size= params['BATCH_SIZE'], verbose=2,
        validation_data= (X_val, y_val)) 

    return history, model

I am calling standard talos scan call:

h = ta.Scan(
    X_train, 
    X_res_train, 
    params= PARAMS_OPTIMIZATION,
    model= LSTMTrainOptimization,
    experiment_name='EXP_NO1',
    x_val=X_val,
    y_val=X_res_val
)

the param variable is:

PARAMS_OPTIMIZATION = {
    'NUM_NEURONS_FIRST_LAYER': [10, 8, 6], 'NUM_NEURONS_SECOND_LAYER': [2, 4, 6],
    'BATCH_SIZE': [2, 4, 6], 'EPOCHS': [10, 200, 250], 'LOOK_BACK': [6, 9, 12],
    'DROP_OUT': [0, 0.2], 'FORWARD_DAYS': [1, 3, 6], 'LOSS': ['logcosh', keras.losses.binary_crossentropy],
    'OPTIMIZER': ['Adam', "RMSprop"],
    'DENSE_ACTIVATION': ['relu', 'elu']
}

Does that affect the result of the optimization?

github-actions[bot] commented 3 years ago

Welcome to Talos community! Thanks so much for creating your first issue :)

mikkokotila commented 3 years ago

Can you share the error message you get.

mikkokotila commented 3 years ago

Your inputs to Talos appear to be wrong. Have a look at docs. For example, see how you would want to handle multi-input models from docs.