search

autonomio / talos

Hyperparameter Experiments with TensorFlow and Keras
https://autonom.io
MIT License
1.62k stars 268 forks source link

Issue with results when using ta.Depoly() and ta.Restore() #241

Closed 17laker closed 5 years ago

17laker commented 5 years ago

Sorry to ask for your help all the time... I really appreciate your help. Thanks!

Here is the thing. The results of hyperparameter tuning was pretty good. The val_loss and loss was satisfactory. After doing hyperparamter tuning I tried to use ta.Depoly() and ta.Restore() to predict output in a test data set. However, I found that runing the below codes would return totally uncorrect results which are totally different from actual values. As the below codes show, I also tried traning data set and validation data set but I still got uncorrect results which would cause val_loss and loss very large. I am wondering am I using wrong codes? Do you have an idea?

Here are the codes. (The codes about hyperparameter tuning worked so don't show them.)

ta.Deploy(h,'model_1',metric = 'val_loss'); model_1 = ta.Restore('model_1.zip') Y_train_hat = model_1.model.predict(X_train) Y_test_hat = model_1.model.predict(X_test) Y_val_hat = model_1.model.predict(X_val)

mikkokotila commented 5 years ago

What ta.Deploy() does is pick one model from the many that result in ta.Scan(). You are asking to save the "best model" based on loss function, but you are not setting asc=True which means that the best model will be the worst model (highest loss).

mikkokotila commented 5 years ago

Closing here, feel free to open new issues is something arise.

mikkokotila commented 5 years ago

I've added docstring to Deploy(). Make sure to check it also.

17laker commented 5 years ago

Sorry for the late reply. I will check the results again with the use of asc=Ture. Thank you very much!

17laker commented 5 years ago

You are right. It's my fault! Thank you very much! Thank you!