ymattu / MlBayesOpt

R package to tune parameters for machine learning(Support Vector Machine, Random Forest, and Xgboost), using bayesian optimization with gaussian process
Other
45 stars 15 forks source link

Error in GP_deviance(param_init_200d[i, ], X, Y, nug_thres, corr = corr) : Infinite values of the Deviance Function, unable to find optimum parameters #58

Open ghost opened 6 years ago

ghost commented 6 years ago

When using the xgb_cv_opt function I get the following error after four rounds.

Error in GP_deviance(param_init_200d[i, ], X, Y, nug_thres, corr = corr) : Infinite values of the Deviance Function, unable to find optimum parameters

I found this page

https://github.com/yanyachen/rBayesianOptimization/issues/36

talking about the same error in the rbayesianoptimization package and tried playing around with the parameter ranges but still cannot get the optimization to run. Reproducible example below.

df_example <- read.csv(textConnection("V2,V3,V4,V5,V6,V7,V8,V9,V10,V11,V12,V13,V14,V15,V16,V17,V18,V19,V20,V21,V22,V23,V24,V25,V26,V27,V28,V29,V30,V31,V32,V33,V34,V35,V36,V37,V38,V39,V40,V41,V42,V43,V44,V45,V46,V47,V48,V49,V50,V51,V52,V53,V54,V55,V56,V57,V58,V59,V60,V61,V62,V63,V64,V65,V66,V67,V68,V69,V70,V71,V72,V73,y
                        1.027244757,-0.362509685,-0.14182585,0,1,0,1,1,0,0,0.098083245,0.155305631,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,1
                        0.140333592,-0.362509685,-0.14182585,0,1,0,1,1,0,0,0.294338347,0.066774355,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.511186324,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,0.437868197,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1
                        1.027244757,-0.362509685,-0.14182585,0,1,0,0,0,0,1,1.205313519,0.39842224,1,0,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,0.644375431,-0.068217098,0,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.288479985,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.171401372,-0.068217098,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,1.427931246,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,1
                        -2.165635439,-0.362509685,0.2818476,1,0,1,0,0,0,0,0.013137007,-0.068217098,0,1,0,1,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0
                        -0.923959807,1.77708902,-0.05709116,0,0,0,0,0,0,0,-0.374979425,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.670826667,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0
                        1.027244757,3.916687725,7.56903095,0,0,0,0,0,0,1,-0.875869311,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,1,0,-0.711835196,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.719069536,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.960815549,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1
                        -0.923959807,-0.362509685,-0.05709116,0,0,0,0,0,0,1,0.188887844,-0.068217098,0,0,1,0,1,0,0,0,0,1,0,0,0,1,1,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0
                        1.027244757,1.77708902,0.536051671,0,0,0,0,0,0,0,-0.650322403,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.44674435,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,1
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.979766611,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,3.725873442,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.952028007,-0.068217098,1,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0
                        -0.923959807,1.77708902,-0.22656054,0,0,0,0,0,0,1,1.323945334,-0.068217098,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.515580095,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1
                        1.027244757,1.77708902,0.02764353,0,1,0,0,0,0,1,-0.53901354,-0.030471007,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.273922694,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.744056183,-0.068217098,0,0,1,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0
                        0.140333592,-0.362509685,-0.05709116,0,0,0,0,0,0,1,-0.735268641,-0.068217098,0,0,1,0,0,0,1,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,2.809039908,-0.068217098,0,0,1,1,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1
                        -2.165635439,-0.362509685,-0.22656054,0,0,1,0,0,0,0,-0.976926042,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0
                        0.140333592,1.77708902,-0.22656054,0,0,0,0,0,0,1,-0.751379134,-0.068217098,0,0,1,0,0,0,1,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,1,0,1.262432541,-0.068217098,0,1,0,1,0,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        0.140333592,1.77708902,-0.05709116,0,0,0,0,0,0,1,-0.328112535,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,0,1
                        -2.165635439,-0.362509685,-0.22656054,0,0,1,0,0,0,0,0.427616065,-0.068217098,1,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.380837786,-0.068217098,0,1,0,0,0,0,1,0,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.300285319,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.650322403,-0.068217098,1,0,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.335435487,-0.068217098,1,0,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        0.140333592,1.77708902,0.11237822,0,0,0,0,1,0,1,-0.654716174,0.954291261,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.809962747,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,1,1,-0.177259733,-0.068217098,0,1,0,0,1,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,1
                        1.027244757,-0.362509685,0.705521051,0,1,0,1,1,1,1,0.54917706,1.999725356,1,0,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,1.318086973,-0.068217098,0,1,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.679614209,-0.068217098,1,0,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0
                        1.027244757,-0.362509685,-0.14182585,0,1,0,1,1,0,1,0.467160003,0.206564964,0,0,1,0,0,0,1,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.14182585,0,0,0,0,0,0,1,2.415065115,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0
                        -2.165635439,-0.362509685,0.11237822,0,0,0,1,1,0,0,-0.625424368,0.012099869,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.448120329,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.73087487,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.814267906,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.411594183,-0.068217098,0,0,1,0,0,1,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.246095478,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.291497777,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.714764377,-0.068217098,0,1,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,1
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.58880961,-0.068217098,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,0.11237822,0,1,0,0,0,1,0,-0.672291258,-0.035109976,0,1,0,0,1,0,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.908090298,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.301749909,-0.068217098,0,0,1,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.376444015,-0.068217098,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0
                        1.027244757,1.77708902,-0.05709116,0,1,0,0,0,0,1,-0.44527976,-0.043375544,0,1,0,1,0,0,0,0,0,0,1,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1
                        1.027244757,-0.362509685,2.230745473,0,1,0,0,1,1,1,0.981231202,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,0,0,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,1,0,0,1,0,-0.708906016,-0.068217098,0,1,0,0,1,0,0,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,1,0,0,-0.125999072,-0.065276094,0,0,1,1,0,0,0,0,0,0,1,0,1,0,1,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,1.647619793,-0.068217098,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.44674435,-0.068217098,0,1,0,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.194746205,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.985713584,-0.068217098,0,0,1,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,1
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.326647945,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.973996862,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,1
                        -0.923959807,1.77708902,-0.22656054,0,0,1,0,0,0,1,-0.477500747,-0.068217098,0,0,1,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.673755848,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,1,0,0,0,0,-0.912484069,-0.068217098,1,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0
                        0.140333592,1.77708902,-0.22656054,0,1,0,0,0,1,0,-0.407200412,-0.050590295,0,1,0,0,0,0,1,1,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.669362077,-0.068217098,0,1,0,0,0,0,1,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0
                        0.140333592,-0.362509685,-0.14182585,0,0,0,0,0,0,0,1.366418453,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.878798491,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0
                        1.027244757,1.77708902,1.976541403,0,0,0,0,0,0,1,-0.92127161,-0.068217098,0,0,1,1,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0
                        1.027244757,1.77708902,0.536051671,0,0,0,0,0,0,0,0.21525047,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.464319434,-0.068217098,0,1,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0
                        1.027244757,-0.362509685,-0.14182585,0,1,0,1,0,0,0,-0.679614209,-0.015445615,0,0,1,0,1,0,0,0,0,0,1,0,0,1,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,0.506703941,-0.068217098,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,1
                        -2.165635439,-0.362509685,-0.22656054,0,1,1,0,1,0,0,-0.54194272,-0.041478948,1,0,0,0,1,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0
                        -2.165635439,1.77708902,-0.22656054,0,0,1,0,0,0,0,-0.046911196,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,1,-0.320789583,-0.068217098,0,0,1,0,0,0,1,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0
                        1.027244757,1.77708902,-0.22656054,0,0,0,0,0,0,0,0.194746205,-0.068217098,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,1
                        1.027244757,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.199139976,-0.068217098,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,1,0,0.883103651,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0
                        -2.165635439,-0.362509685,-0.22656054,0,0,1,0,0,0,0,-0.417452544,-0.068217098,1,0,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0
                        0.140333592,1.77708902,0.02764353,0,0,0,0,0,0,1,-0.300285319,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,0,1,1,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0
                        -2.165635439,1.77708902,0.19711291,0,0,1,0,0,0,0,-0.72794569,-0.068217098,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0,1,0
                        0.140333592,1.77708902,-0.22656054,0,0,0,0,0,0,1,-0.997430306,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,1
                        -2.165635439,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.890515214,-0.068217098,0,0,1,1,0,0,0,0,0,1,0,1,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0
                        -0.923959807,3.916687725,5.365929007,0,0,0,0,0,0,1,-0.755772905,-0.068217098,0,0,1,1,0,0,0,0,0,0,1,0,1,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,0,1
                        1.027244757,-0.362509685,8.585847231,0,0,0,0,0,0,0,-0.719158148,-0.068217098,0,0,1,1,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0
                        0.140333592,6.056286429,-0.22656054,0,0,0,0,0,0,1,1.407426982,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,1,0,0,0,0,0,1,0,0,1
                        -0.923959807,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.924200791,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0
                        1.027244757,-0.362509685,-0.22656054,0,0,0,1,0,0,0,-0.995965716,-0.068217098,0,0,1,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1
                        0.140333592,-0.362509685,-0.14182585,0,0,0,0,0,0,0,-0.183118094,-0.068217098,0,1,0,1,0,0,0,0,0,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,0,-0.042517425,-0.068217098,0,0,1,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,1,0
                        0.140333592,-0.362509685,-0.22656054,0,0,0,0,0,0,1,0.077578981,-0.068217098,0,1,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,1,1,0,0,0,0,1,0,0,0,0,0,1,0,0,1"
))
set.seed(71)    

res0 <- xgb_cv_opt(data = df_example,
                   label = y,
                   objectfun = "binary:logistic",
                   evalmetric = "auc",
                   n_folds = 3,
                   classes = 10,
                   init_points = 4,
                   n_iter = 5)
ymattu commented 6 years ago

I couldn't reproduce the error, as following.

> set.seed(71)
> res0 <- xgb_cv_opt(data = df_example,
+                    label = y,
+                    objectfun = "binary:logistic",
+                    evalmetric = "auc",
+                    n_folds = 3,
+                    classes = 10,
+                    init_points = 4,
+                    n_iter = 5)
elapsed = 0.01  Round = 1   eta_opt = 0.4475    max_depth_opt = 5.0000  nrounds_opt = 86.7596   subsample_opt = 0.1971  bytree_opt = 0.7882 Value = 0.5852 
elapsed = 0.01  Round = 2   eta_opt = 0.1121    max_depth_opt = 5.0000  nrounds_opt = 144.4636  subsample_opt = 0.7513  bytree_opt = 0.8698 Value = 0.6113 
elapsed = 0.01  Round = 3   eta_opt = 0.4441    max_depth_opt = 5.0000  nrounds_opt = 130.1620  subsample_opt = 0.4701  bytree_opt = 0.7318 Value = 0.5542 
elapsed = 0.01  Round = 4   eta_opt = 0.8827    max_depth_opt = 5.0000  nrounds_opt = 141.4816  subsample_opt = 0.8389  bytree_opt = 0.7178 Value = 0.5254 
elapsed = 0.01  Round = 5   eta_opt = 0.1328    max_depth_opt = 5.0000  nrounds_opt = 104.9175  subsample_opt = 0.3617  bytree_opt = 0.8559 Value = 0.5064 
elapsed = 0.01  Round = 6   eta_opt = 0.8979    max_depth_opt = 5.0000  nrounds_opt = 153.8820  subsample_opt = 0.1759  bytree_opt = 0.8697 Value = 0.5993 
elapsed = 0.01  Round = 7   eta_opt = 0.4374    max_depth_opt = 5.0000  nrounds_opt = 141.3386  subsample_opt = 0.1684  bytree_opt = 0.8699 Value = 0.3771 
elapsed = 0.01  Round = 8   eta_opt = 0.7030    max_depth_opt = 4.0000  nrounds_opt = 144.5133  subsample_opt = 0.4369  bytree_opt = 0.8664 Value = 0.5131 
elapsed = 0.02  Round = 9   eta_opt = 0.6536    max_depth_opt = 6.0000  nrounds_opt = 147.7658  subsample_opt = 0.7710  bytree_opt = 0.8733 Value = 0.6066 

 Best Parameters Found: 
Round = 2   eta_opt = 0.1121    max_depth_opt = 5.0000  nrounds_opt = 144.4636  subsample_opt = 0.7513  bytree_opt = 0.8698 Value = 0.6113 

I wonder that error is because of random seed or something... If you still get error, try to increase the kappa parameter of acquisition function.

Infinite values of the Deviance Function

This error often means the searching parameter fails because of local maximum point. Increasing kappa enables to enlarge the parameter space to search.

ghost commented 6 years ago

Strange, when I run the code I'm still getting the exact same error after four rounds. I tried multiple seeds, and increasing kappa but nothing seems to be working.

I had a friend run the code and he got the same error after four rounds. We are both running R version 3.5.1 and have fresh installs of MlBaysOpt

ghost commented 6 years ago

Here's my session info

R version 3.5.1 (2018-07-02)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] MlBayesOpt_0.3.3

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.18                rstudioapi_0.7             
 [3] bindr_0.1.1                 magrittr_1.5               
 [5] GPfit_1.0-0                 tidyselect_0.2.4           
 [7] lattice_0.20-35             R6_2.2.2                   
 [9] rlang_0.2.1                 foreach_1.4.4              
[11] dplyr_0.7.6                 tools_3.5.1                
[13] grid_3.5.1                  data.table_1.11.4          
[15] e1071_1.7-0                 class_7.3-14               
[17] iterators_1.0.10            yaml_2.2.0                 
[19] xgboost_0.71.2              assertthat_0.2.0           
[21] tibble_1.4.2                crayon_1.3.4               
[23] Matrix_1.2-14               bindrcpp_0.2.2             
[25] purrr_0.2.5                 lhs_0.16                   
[27] codetools_0.2-15            glue_1.3.0                 
[29] stringi_1.1.7               compiler_3.5.1             
[31] pillar_1.3.0                ranger_0.10.1              
[33] rBayesianOptimization_1.1.0 pkgconfig_2.0.1 `
ymattu commented 6 years ago

Sorry, I tried in old version of R (3.4.4). When I tried in R 3.5.1, I got an error, but the error occurred after 7 rounds and the error message was different.

Error in { : task 29 failed - "non-finite value supplied by optim"
Calls: xgb_cv_opt ... Utility_Max -> %>% -> eval -> eval -> %do% -> <Anonymous>

My main machine is Mac, so I'll research on Windows. Please wait a minute.

TheXu commented 5 years ago

I had the same error.

What worked for me was to change the acquisition function from Expected improvement to GP lower confidence bound.

orrymr commented 5 years ago

I too am getting a similar error: Error in { : task 64 failed - "non-finite value supplied by optim". I am running Windows, R version 3.5.2.

Changing acq = "ucb" seems to fix problem... (granted, it changes the acquisition function).