Open badfish2019 opened 3 months ago
best_model = tuner.get_best_models(num_models=1) best_hp = tuner.get_best_hyperparameters()[0]
hypermodel = MyHyperModel()
model = hypermodel.build(best_hp)
hypermodel.fit( best_hp, model, training_data=(X_train, y_train), validation_data=(X_val, y_val), epochs=200, )
when I use the best_hp to train a model,the performance of the model is difference compare to the best_model . why? this is best_model
Train: {'MSE': '0.0224', 'MAE': '0.1155', 'RMSE': '0.1498', 'MAPE': '4.0837', 'R2': '0.9974', 'Corr.': '0.9992'} Val: {'MSE': '0.0030', 'MAE': '0.0406', 'RMSE': '0.0552', 'MAPE': '4.1977', 'R2': '0.8491', 'Corr.': '0.9227'} Test: {'MSE': '0.8920', 'MAE': '0.5601', 'RMSE': '0.9444', 'MAPE': '211.2147', 'R2': '0.9391', 'Corr.': '0.9744'}
this is best_hp (retrain model)
Train: {'MSE': '0.0179', 'MAE': '0.1021', 'RMSE': '0.1338', 'MAPE': '3.0199', 'R2': '0.9979', 'Corr.': '0.9995'} Val: {'MSE': '0.0154', 'MAE': '0.1044', 'RMSE': '0.1242', 'MAPE': '11.0382', 'R2': '0.2350', 'Corr.': '0.8447'} Test: {'MSE': '1.4758', 'MAE': '0.7471', 'RMSE': '1.2148', 'MAPE': '261.5144', 'R2': '0.8992', 'Corr.': '0.9585'}
Same problem here. Any ideas?
best_model = tuner.get_best_models(num_models=1) best_hp = tuner.get_best_hyperparameters()[0]
hypermodel = MyHyperModel()
model = hypermodel.build(best_hp)
hypermodel.fit( best_hp, model, training_data=(X_train, y_train), validation_data=(X_val, y_val), epochs=200, )
when I use the best_hp to train a model,the performance of the model is difference compare to the best_model . why? this is best_model
Train: {'MSE': '0.0224', 'MAE': '0.1155', 'RMSE': '0.1498', 'MAPE': '4.0837', 'R2': '0.9974', 'Corr.': '0.9992'} Val: {'MSE': '0.0030', 'MAE': '0.0406', 'RMSE': '0.0552', 'MAPE': '4.1977', 'R2': '0.8491', 'Corr.': '0.9227'} Test: {'MSE': '0.8920', 'MAE': '0.5601', 'RMSE': '0.9444', 'MAPE': '211.2147', 'R2': '0.9391', 'Corr.': '0.9744'}
this is best_hp (retrain model)
Train: {'MSE': '0.0179', 'MAE': '0.1021', 'RMSE': '0.1338', 'MAPE': '3.0199', 'R2': '0.9979', 'Corr.': '0.9995'} Val: {'MSE': '0.0154', 'MAE': '0.1044', 'RMSE': '0.1242', 'MAPE': '11.0382', 'R2': '0.2350', 'Corr.': '0.8447'} Test: {'MSE': '1.4758', 'MAE': '0.7471', 'RMSE': '1.2148', 'MAPE': '261.5144', 'R2': '0.8992', 'Corr.': '0.9585'}