search

yanyachen / rBayesianOptimization

Bayesian Optimization of Hyperparameters
81 stars 21 forks source link

What is the "Pred" component of FUN for? #22

Closed MichaelChirico closed 6 years ago

MichaelChirico commented 6 years ago

Perhaps I'm missing something since I'm new to BO, but it's not clear to me from the documentation what the "Pred" component of the return value for FUN is supposed to be. Can I just set it to 0 without consequence?

Examining the code, it appears it's not particularly used:

# It's stored in Pred_list in two steps:
Pred_list[[i]] <- This_Score_Pred$Pred
Pred_list[[j]] <- Next_Score_Pred$Pred

# And Pred_list is simply converted to a data.table
#  (side note -- setDT works on lists, so is probably preferable)
Pred_DT <- data.table::as.data.table(Pred_list)

# Pred_DT is then stored in Result
Result <- list(Best_Par = Best_Par, Best_Value = Best_Value, 
               History = DT_history, Pred = Pred_DT)

# and finally Result is returned
return(Result)

Thus, besides storing this value, it doesn't seem to be worth anything? What am I missing?

yanyachen commented 6 years ago

It's not something specific to BO. ThePred should be used for storing prediction for different sets of hyper-parameters. The goal is to make it convenient for people who trying to find diversity of different hyper-parameters and doing ensemble/stacking later. If you are only interested in optimizing the performance of your current model, then you can use 0 as placeholder without any impact.

MichaelChirico commented 6 years ago

OK, thanks. Something in the examples clarifying this/using Pred could go a long way.