Parameters visiting.param and simple.function were missing
Also set trace.mat = FALSE because we dont' need this
Added a comment on smooth in the docs because most of the time you may want this to be FALSE (but GenSA default is TRUE).
Did a small benchmark on xgboost comparing some optimizers on 2D, 3D, 5D problems on 10 tasks and default GenSA performs very poorly (y-axis is mean normalized regret over the mean of the 10 repls over the 30 problems in total):
Small ablation on 3 of the problems where GenSA performed worst shows that if you set smooth = FALSE you get quite some performance boosts (GenSAv2 or GenSAv4; GenSAv3 is the current default):
The four versions are constructed w.r.t smooth and simple.function (full factorial design: v1 TRUE/TRUE, v2 FALSE/TRUE, v3 TRUE/FALSE, v4 FALSE/FALSE) but simple.function does not have a strong effect.
smooth is not really explained in the docs of GenSA but what happens is:
If FALSE switches the local search algorithm from using L-BFGS-B (default) to Nelder-Mead that works better when the objective function has very few places where numerical derivatives can be computed.
I would say that numerical derivatives are quite hard to compute in HPO problems (noisy, resampling, etc.) and therefore argue to go with smooth = FALSE (in mlr3tuning; here in bbotk I suggest we keep things as before but I added a comment in the docs).
visiting.paramandsimple.functionwere missingtrace.mat = FALSEbecause we dont' need thissmoothin the docs because most of the time you may want this to beFALSE(but GenSA default isTRUE).Did a small benchmark on xgboost comparing some optimizers on 2D, 3D, 5D problems on 10 tasks and default GenSA performs very poorly (y-axis is mean normalized regret over the mean of the 10 repls over the 30 problems in total):
Small ablation on 3 of the problems where GenSA performed worst shows that if you set
smooth = FALSEyou get quite some performance boosts (GenSAv2 or GenSAv4; GenSAv3 is the current default):The four versions are constructed w.r.t
smoothandsimple.function(full factorial design: v1 TRUE/TRUE, v2 FALSE/TRUE, v3 TRUE/FALSE, v4 FALSE/FALSE) butsimple.functiondoes not have a strong effect.smoothis not really explained in the docs of GenSA but what happens is: IfFALSEswitches the local search algorithm from using L-BFGS-B (default) to Nelder-Mead that works better when the objective function has very few places where numerical derivatives can be computed. I would say that numerical derivatives are quite hard to compute in HPO problems (noisy, resampling, etc.) and therefore argue to go withsmooth = FALSE(in mlr3tuning; here in bbotk I suggest we keep things as before but I added a comment in the docs).