Closed pfistfl closed 2 weeks ago
Below a list of proposed callbacks. Some _end, _begin might be superfluous.
_end
_begin
on_optimization_begin: Additional setup, ....
on_optimization_begin
on_optimization_end: Adapt archive before finishing
on_optimization_end
on_design_eval_start: Add e.g. multiple defaults, meta-learned stuff, ...
on_design_eval_start
on_iteration_begin:
on_iteration_begin
on_iteration_end: Track e.g. loss on held-out dataset || Could be done with mlr3
on_iteration_end
on_acquisition_begin:
on_acquisition_begin
on_acquisition_end: Perform some checks on the proposed points before evaluating.
on_acquisition_end
on_eval_begin:
on_eval_begin
Can I change the AcqFunction hyperparameters depending on some schedule?
AcqFunction
Can I change AcqFunction|Surrogate optimizer after some amount of iterations?
Surrogate
How would I weave in a random eval every nth iteration?
How can I add cheap sub-evals (c.f. classification thresholds, nrounds, ...) into optimization?
Can I terminate optimization if some non-terminator criterion I define e.g. in the acqfun is met?
Can I just debug at a given LOC? i.e. simply adding a callback that calls browser()?
browser()
...
See also mlr-org/bbotk#180
to the largest extent solved by bbotk callbacks and #152 now
Below a list of proposed callbacks. Some
_end
,_begin
might be superfluous.on_optimization_begin
: Additional setup, ....on_optimization_end
: Adapt archive before finishingon_design_eval_start
: Add e.g. multiple defaults, meta-learned stuff, ...on_iteration_begin
:on_iteration_end
: Track e.g. loss on held-out dataset || Could be done with mlr3on_acquisition_begin
:on_acquisition_end
: Perform some checks on the proposed points before evaluating.on_eval_begin
:Can I change the
AcqFunction
hyperparameters depending on some schedule?Can I change
AcqFunction
|Surrogate
optimizer after some amount of iterations?How would I weave in a random eval every nth iteration?
How can I add cheap sub-evals (c.f. classification thresholds, nrounds, ...) into optimization?
Can I terminate optimization if some non-terminator criterion I define e.g. in the acqfun is met?
Can I just debug at a given LOC? i.e. simply adding a callback that calls
browser()
?...