Callbacks

[pfistfl]

Below a list of proposed callbacks. Some _end, _begin might be superfluous.

• on_optimization_begin: Additional setup, ....

• on_optimization_end: Adapt archive before finishing

• on_design_eval_start: Add e.g. multiple defaults, meta-learned stuff, ...

• on_iteration_begin:

• on_iteration_end: Track e.g. loss on held-out dataset || Could be done with mlr3

• on_acquisition_begin:

• on_acquisition_end: Perform some checks on the proposed points before evaluating.

• on_eval_begin:

• Can I change the AcqFunction hyperparameters depending on some schedule?

• Can I change AcqFunction|Surrogate optimizer after some amount of iterations?

• How would I weave in a random eval every nth iteration?

• How can I add cheap sub-evals (c.f. classification thresholds, nrounds, ...) into optimization?

• Can I terminate optimization if some non-terminator criterion I define e.g. in the acqfun is met?

• Can I just debug at a given LOC? i.e. simply adding a callback that calls browser()?

• ...

[sumny]

See also mlr-org/bbotk#180

[sumny]

to the largest extent solved by bbotk callbacks and #152 now