I'm using rBayesianOptimization to find the best combination of hyperparameters to use in a XGBoost model.
From a grid of these hyperparameters, i define a function (object "FUN") that runs a xgb.cv() over them, and then return the maximum value of the AUC in the test dataset, which will be maximized by the BayesianOptimization() function (The "Score" element).
So, the final object, after i run BayesianOptimization() function, will have the list of each combination of hyparameters tested, and the "Value", which is the maximum of the AUC found in the test dataset over the iterations of xgb.cv() for each combination, as i defined.
I would like to have the AUC of the train dataset associated with the maximum of the AUC in the test dataset, to analyze overfitting. Is that possible?
I have tried to add an element in the list that is returned in the "FUN" function, but it just desappeared in the final object.
How could i store both the test and train performances? (Even if the BayesianOptimization uses just the "Score" to do the optimization).
I'm using rBayesianOptimization to find the best combination of hyperparameters to use in a XGBoost model.
From a grid of these hyperparameters, i define a function (object "FUN") that runs a xgb.cv() over them, and then return the maximum value of the AUC in the test dataset, which will be maximized by the BayesianOptimization() function (The "Score" element).
So, the final object, after i run BayesianOptimization() function, will have the list of each combination of hyparameters tested, and the "Value", which is the maximum of the AUC found in the test dataset over the iterations of xgb.cv() for each combination, as i defined.
I would like to have the AUC of the train dataset associated with the maximum of the AUC in the test dataset, to analyze overfitting. Is that possible?
I have tried to add an element in the list that is returned in the "FUN" function, but it just desappeared in the final object.
How could i store both the test and train performances? (Even if the BayesianOptimization uses just the "Score" to do the optimization).
Hope my english let me to be clear.
Thanks in advance