search

autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 268 forks source link

TypeError: unsupported operand type(s) for +: 'EarlyStopping' and 'list' #304

Closed kenfus closed 5 years ago

kenfus commented 5 years ago

Thanks so much for coming here to raise an issue. Please take a moment to 'check' the below boxes:

If you still have an error, please submit complete trace and a code with:

You can provide the code in pastebin / gist or any other format you like. This is the whole code. It works when I remove the early_stopping but I'd like to use it!

Thanks so much.

https://towardsdatascience.com/hyperparameter-optimization-with-keras-b82e6364ca53

Function to create model, required for KerasClassifier

def get_model(x_train, y_train, x_val, y_val, params):

create model

#parameters defined
lr = params['lr']
epochs= params['epochs']
neurons = params['neurons']
optimizer = params['optimizer']
loss= params['loss']
decay = params['decay']
momentum=params['momentum']
batch_size = params['batch_size']
dropout_rate = params['dropout']

input_shape = X_train.shape[1]

model = Sequential()
model.add(Dense(neurons, input_shape = (input_shape,))),
model.add(LeakyReLU(alpha=0.01)),
model.add(BatchNormalization()),

model.add(Dropout(dropout_rate, seed=42)),
model.add(Dense(neurons)),
model.add(LeakyReLU(alpha=0.01)),
model.add(BatchNormalization()),

model.add(Dropout(dropout_rate, seed=42)),
model.add(Dense(neurons)),
model.add(LeakyReLU(alpha=0.01)),
model.add(BatchNormalization()),

model.add(Dropout(dropout_rate, seed=42)),
model.add(Dense(neurons)),
model.add(LeakyReLU(alpha=0.01)),
model.add(BatchNormalization()),

model.add(Dropout(dropout_rate, seed=42)),
model.add(Dense(neurons)),
model.add(LeakyReLU(alpha=0.01)),
model.add(BatchNormalization()),

model.add(Dropout(dropout_rate, seed=42)),
model.add(Dense(5, activation='softmax'))
# Compile model

# https://stackoverflow.com/questions/54377668/talosreturnerror-talos-make-sure-that-input-model-returns-out-model-model-f

if optimizer=="Adam":
        opt=keras.optimizers.Adam(lr=lr, beta_1=0.9, beta_2=0.999)
if optimizer=="Adagrad":
        opt=keras.optimizers.Adagrad(lr=lr, epsilon=None, decay=decay)
if optimizer=="sgd":
        opt=keras.optimizers.SGD(lr=lr, momentum=momentum, decay=decay, nesterov=False)

model.compile(loss=loss,optimizer=opt,
             metrics=['accuracy'])

history = model.fit(x = x_train, y= y_train, validation_data=[x_val, y_val], 
                    epochs=params['epochs'], batch_size = batch_size,verbose=1,callbacks=early_stopper(params['epochs']))

return history, model

p = {'lr': [0.0001], 'neurons':[1200,1500], 'decay':[0], 'optimizer': ["Adam"], 'hidden_layers':[3], 'momentum': [0.0], 'batch_size': [128], 'loss': ["sparse_categorical_crossentropy"], 'epochs': [250], 'dropout': [0.2,0.3]}

t = ta.Scan(x=X_train, y=y_train, params = p, model=get_model)

TypeError Traceback (most recent call last)

in () 77 78 ---> 79 t = ta.Scan(x=X_train, y=y_train, params = p, model=get_model) 7 frames /usr/local/lib/python3.6/dist-packages/talos/scan/Scan.py in __init__(self, x, y, params, model, experiment_name, x_val, y_val, val_split, random_method, performance_target, fraction_limit, round_limit, time_limit, boolean_limit, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, minimize_loss, seed, clear_session, disable_progress_bar, print_params, debug) 170 # input parameters section ends 171 --> 172 self.runtime() 173 174 def runtime(self): /usr/local/lib/python3.6/dist-packages/talos/scan/Scan.py in runtime(self) 175 176 from .scan_run import scan_run --> 177 self = scan_run(self) /usr/local/lib/python3.6/dist-packages/talos/scan/scan_run.py in scan_run(self) 24 # otherwise proceed with next permutation 25 from .scan_round import scan_round ---> 26 self = scan_round(self) 27 self.pbar.update(1) 28 /usr/local/lib/python3.6/dist-packages/talos/scan/scan_round.py in scan_round(self) 17 # fit the model 18 from ..model.ingest_model import ingest_model ---> 19 self.model_history, self.keras_model = ingest_model(self) 20 self.round_history.append(self.model_history.history) 21 /usr/local/lib/python3.6/dist-packages/talos/model/ingest_model.py in ingest_model(self) 8 self.x_val, 9 self.y_val, ---> 10 self.round_params) in get_model(x_train, y_train, x_val, y_val, params) 58 59 history = model.fit(x = x_train, y= y_train, validation_data=[x_val, y_val], ---> 60 epochs=params['epochs'], batch_size = batch_size,verbose=1,callbacks=early_stopper(params['epochs'])) 61 62 return history, model /usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs) 1037 initial_epoch=initial_epoch, 1038 steps_per_epoch=steps_per_epoch, -> 1039 validation_steps=validation_steps) 1040 1041 def evaluate(self, x=None, y=None, /usr/local/lib/python3.6/dist-packages/keras/engine/training_arrays.py in fit_loop(model, f, ins, out_labels, batch_size, epochs, verbose, callbacks, val_f, val_ins, shuffle, callback_metrics, initial_epoch, steps_per_epoch, validation_steps) 104 count_mode, 105 stateful_metrics=model.stateful_metric_names)) --> 106 _callbacks += (callbacks or []) + [model.history] 107 callbacks = cbks.CallbackList(_callbacks) 108 out_labels = out_labels or [] TypeError: unsupported operand type(s) for +: 'EarlyStopping' and 'list'
mikkokotila commented 5 years ago

Can you try the upcoming release v.0.6 and see if you get the same error message:

pip install git+https://github.com/autonomio/talos@params-api-test
kenfus commented 5 years ago

I fixed it:

callbacks=[EarlyStopping(monitor='val_acc', patience=int(4))]

It has to be written like this! I found it in the Keras-Manual.

mikkokotila commented 5 years ago

Great :) Closing here.

nijatmursali commented 4 years ago

This mainly happens because you forgot to put [] for the list. In my case:

stopping = callbacks.EarlyStopping(monitor='val_loss', patience=20)
history = model.fit_generator(train_generator,
                              steps_per_epoch=100,
                              epochs=100,
                              callbacks=[stopping],
                              validation_data=test_generator,
                              validation_steps=100)

Hope it helps.

fffii commented 1 year ago

This mainly happens because you forgot to put [] for the list. In my case:

stopping = callbacks.EarlyStopping(monitor='val_loss', patience=20)
history = model.fit_generator(train_generator,
                              steps_per_epoch=100,
                              epochs=100,
                              callbacks=[stopping],
                              validation_data=test_generator,
                              validation_steps=100)

Hope it helps.

Thank you very much for your answer! It really helps me