search

mlr-org / mlr3hyperband

Successive Halving and Hyperband in the mlr3 ecosystem
https://mlr3hyperband.mlr-org.com
GNU Lesser General Public License v3.0
18 stars 5 forks source link

Option that additional Measures that are not optimized over, but only measured #47

Closed juliambr closed 4 years ago

juliambr commented 4 years ago

If I pass k > 1 measures to the tuning instance, it gets automatically treated as a multi-objective problem by hyperband with k objectives.

We eventually want to optimize w.r.t. l < k measures only, but just log another measure.

Example: For hyperband, we might want to optimize for a performance measure, but log the training time for the configurations to see how the budget parameter influences our actual resource, the training time.

jakob-r commented 4 years ago

It is not possible with the actual mlr3tuning/bbotk design. However you can calculate most measures afterwards on the resample results that are stored in the archive.