Closed parindam closed 8 years ago
Hi,
Thanks for reporting this. Would you mind sharing the complete example, so I can examine?
I suspect you get the error, because you don't properly wrap your model into a function. If, in your example, you pass model
directly into optim.minimizer, it will only see the first line, i.e.
model = Sequential()
which is clearly lacking input_dim
or input_shape
. Properly done, it should look as follows:
def return_full_model_here():
import everything_you_need_for_the_model
X = ...
y = ...
model = Sequential()
model.add(LSTM(input_dim=388, output_dim=300, return_sequences=False))
....
model.add(Activation("linear"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
model.fit(X, y)
return {'loss': model_metric_to_minimize, 'status': STATUS_OK}
You need all this, since the model with double brackets dropped-in clearly isn't valid code in any sense and all hyperas does is tell hyperopt what to plug in, so to make this an actual keras model. Therefore the function wrapper.
I hope this makes sense, probably I should update the documentation accordingly.
This is the code i have which is throwing error. sometimes, it also throws error saying, "OSError: [Errno 2] No such file or directory: './temp.pyc'" It just works fine without {{...}} and if i call keras_model() directly.
def keras_model():
from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
from keras.callbacks import EarlyStopping, ModelCheckpoint
model = Sequential()
model.add(LSTM(input_dim=388, output_dim=300, return_sequences=False))
model.add(Dropout({{uniform(0, 1)}}))
model.add(Dense(output_dim=1))
model.add(Activation("linear"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")
early_stopping = EarlyStopping(monitor='val_loss', patience=4)
checkpointer = ModelCheckpoint(filepath='keras_weights.hdf5',
verbose=1,
save_best_only=True)
hist = model.fit(X_train, y_train,
batch_size={{choice([32, 64, 128])}},
#batch_size=128,
nb_epoch=1,
validation_split=0.08,
show_accuracy=True)
#callbacks=[early_stopping, checkpointer])
score = model.evaluate(X_test, y_test,
show_accuracy=True, verbose=0)
print('Test accuracy:', score[1])
return {'loss': -score[1], 'status': STATUS_OK}
if __name__ == '__main__':
best_run = optim.minimize(keras_model,
algo=tpe.suggest,
max_evals=10,
trials=Trials())
print(best_run)
I have the same error as @parindam "OSError: [Errno 2] No such file or directory: './temp.pyc". I am using a virtualenv to run keras. And nothing fancy from the code. Do you have an idea ??
As I don't have access to your data, I modified the keras imdb lstm example and reused every component in your example. It works. Have a try at this:
from __future__ import print_function
from hyperopt import Trials, STATUS_OK, tpe
from hyperas import optim
from hyperas.distributions import choice, uniform
def keras_model():
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers.embeddings import Embedding
from keras.layers.recurrent import LSTM
from keras.datasets import imdb
from keras.callbacks import EarlyStopping, ModelCheckpoint
max_features = 20000
maxlen = 100
print('Loading data...')
(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features, test_split=0.2)
print(len(X_train), 'train sequences')
print(len(X_test), 'test sequences')
print("Pad sequences (samples x time)")
X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
print('X_train shape:', X_train.shape)
print('X_test shape:', X_test.shape)
print('Build model...')
model = Sequential()
model.add(Embedding(max_features, 128, input_length=maxlen))
model.add(LSTM(128))
model.add(Dropout({{uniform(0, 1)}}))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
class_mode="binary")
early_stopping = EarlyStopping(monitor='val_loss', patience=4)
checkpointer = ModelCheckpoint(filepath='keras_weights.hdf5',
verbose=1,
save_best_only=True)
hist = model.fit(X_train, y_train,
batch_size={{choice([32, 64, 128])}},
# batch_size=128,
nb_epoch=1,
validation_split=0.08,
show_accuracy=True,
callbacks=[early_stopping, checkpointer])
score, acc = model.evaluate(X_test, y_test, show_accuracy=True, verbose=0)
print('Test accuracy:', acc)
return {'loss': -acc, 'status': STATUS_OK}
if __name__ == '__main__':
best_run = optim.minimize(keras_model,
algo=tpe.suggest,
max_evals=10,
trials=Trials())
print(best_run)
I will publish the above as an example for hyperas. Other than that, I'm afraid right now I'm running out of ideas how to help you.
@parindam your error might be related with keras
rather than hyperas
. There are some requirements regarding the first layer in the model, especially if it's a recurrent one and it also depends on the backend you're using. Quote:
TensorFlow warning For the time being, when using the TensorFlow backend, the number of timesteps used must be specified in your model. Make sure to pass an input_length int argument to your recurrent layer (if it comes first in your model), or to pass a complete input_shape argument to the first layer in your model otherwise.
This might be the issue. Try to have a read here: http://keras.io/layers/recurrent/
I copy/paste your new example with LSTM and i have the following error.
IndexError Traceback (most recent call last) ---> 64 trials=Trials())
IndexError: list index out of range
@jesuisnicolasdavid it works for me. Make sure you use the latest keras
version, I'm working on the latest master.
keras 0.3.2 is recommended. Well, "works on my machine". I'm still puzzled why this won't work for you.
Ok, i manage to get rid of the previous error with a lot of update. Now i am trying to run the CNN_LSTM example from keras and the error i get is :
IndexError Traceback (most recent call last) ---> 84 trials=Trials()) IndexError: invalid index to scalar variable.
code :
from __future__ import print_function
from hyperopt import Trials, STATUS_OK, tpe
from hyperas import optim
from hyperas.distributions import choice, uniform
def keras_model():
import numpy as np
np.random.seed(1337) # for reproducibility
from keras.preprocessing import sequence
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers.embeddings import Embedding
from keras.layers.recurrent import LSTM, GRU, SimpleRNN
from keras.layers.convolutional import Convolution1D, MaxPooling1D
from keras.datasets import imdb
# Embedding
max_features = 20000
maxlen = 100
embedding_size = 300
# Convolution
filter_length = 6
nb_filter = 64
pool_length = 4
# LSTM
lstm_output_size = 100
# Training
batch_size = 60
nb_epoch = 2
print('Loading data...')
(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features, test_split=0.2)
print(len(X_train), 'train sequences')
print(len(X_test), 'test sequences')
print('Pad sequences (samples x time)')
X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
print('X_train shape:', X_train.shape)
print('X_test shape:', X_test.shape)
model = Sequential()
model.add(Embedding(max_features, embedding_size, input_length=maxlen))
model.add(Dropout({{uniform(0, 1)}}))
model.add(Convolution1D(nb_filter=nb_filter,
filter_length=filter_length,
border_mode='valid',
activation='relu',
subsample_length=1))
model.add(MaxPooling1D(pool_length=pool_length))
model.add(LSTM(lstm_output_size))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
class_mode='binary')
print('Train...')
model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=nb_epoch,
validation_data=(X_test, y_test), show_accuracy=True)
score, acc = model.evaluate(X_test, y_test, batch_size=batch_size,
show_accuracy=True)
print('Test score:', score)
print('Test accuracy:', acc)
return {'loss': -score[1], 'status': STATUS_OK}
if __name__ == '__main__':
best_run = optim.minimize(keras_model,
algo=tpe.suggest,
max_evals=10,
trials=Trials())
print(best_run)
Again, this one works for me as well. I will add another example to address it soon. I really want to help, and trust you see that, but I'm not sure this is really the scope of what was issued originally. Any thoughts?
For me, the first hyperas evaluation is working well, it is when it passes on the second one that it crashes (just after the print of the score and accuracy). Did you run it for at least two evaluations ? If it is the case, maybe i have another issue with some import. Thanks for the help!
Next time, could you please provide the full stack trace? Thanks.
Traceback (most recent call last):
File "examples/cnn_lstm.py", line 77, in <module>
trials=Trials())
File "/usr/local/lib/python2.7/site-packages/hyperas-0.0.2-py2.7.egg/hyperas/optim.py", line 74, in minimize
best = fmin(keras_fmin_fnct, space=get_space(), algo=algo, max_evals=max_evals, trials=trials)
File "/usr/local/lib/python2.7/site-packages/hyperopt/fmin.py", line 334, in fmin
rval.exhaust()
File "/usr/local/lib/python2.7/site-packages/hyperopt/fmin.py", line 294, in exhaust
self.run(self.max_evals - n_done, block_until_done=self.async)
File "/usr/local/lib/python2.7/site-packages/hyperopt/fmin.py", line 268, in run
self.serial_evaluate()
File "/usr/local/lib/python2.7/site-packages/hyperopt/fmin.py", line 187, in serial_evaluate
result = self.domain.evaluate(spec, ctrl)
File "/usr/local/lib/python2.7/site-packages/hyperopt/fmin.py", line 114, in evaluate
rval = self.fn(pyll_rval)
File "./temp_model.py", line 67, in keras_fmin_fnct
IndexError: invalid index to scalar variable.
This indicates an error that arises from treating scalars as iterables.
When you write
score, acc = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True)
instead of
score = model.evaluate(X_test, y_test, batch_size=batch_size, show_accuracy=True)
then
return {'loss': -score[1], 'status': STATUS_OK}
does not make sense. One of those classic copy and paste things I guess.
I'm going to close the Q&A now. ;)
That was a really bad error from me! Thanks for your answer and your help ! You are doing an amazing work!
if I have a model not sequential, how can run hyperopt??
Hi,
I am using keras 0.3.2 and trying a simple LSTM model with hyperas. But I get the following error. Anything wrong ?
Exception: Layer is not connected. Did you forget to set "input_shape"?
model = Sequential()
model.add(LSTM(input_dim=388, output_dim=300, return_sequences=False)) model.add(Dropout({{uniform(0, 1)}})) model.add(Dense(output_dim=1))
model.add(Activation("linear"))
model.compile(loss="mean_squared_error", optimizer="rmsprop")