search

automl / BOAH

BOAH: Bayesian Optimization & Analysis of Hyperparameters
https://www.automl.org
Apache License 2.0
67 stars 8 forks source link

Questions for BOHB paper #8

Open yjhong89 opened 4 years ago

yjhong89 commented 4 years ago

Hi.

I am YJ Hong and I am likeli to use automl's BOAH package for my project.

I have few minor questions while reading BOAB paper.

  1. In 5p, second column, there are some notations which hadn't been specified.

    • What is 'q' in second paragraph?
    • What is 'm' in last paragraph?

  2. What is D_b? I am not getting ' the number of observations for budget b'

Can you explain to me please?

Thank you.

mlindauer commented 4 years ago

Hi,

In general, I would recommend to ask detailed questions regarding BOHB in the corresponding repo: https://github.com/automl/HpBandSter

But let me give it a try:

'q' is a percentile for discriminating the good from the bad points (e.g.,, hyperparameter configurations). For example, if q is 0.1, BOHB uses the 10% best points for building the KDE model of the well-performing area.

'm' seems to be constant which depends on the setting in SH. (not really important since it only says that BOHB is not much slower than random sampling if everything goes south)

'D_b': Since BOHB uses successive halving, we have at each budget (e.g., 1/3 of the total number of training epochs) observations on how well a hyperparameter configuration performed so far (e.g. in terms of validation loss).

Best, Marius

yjhong89 commented 4 years ago

@mlindauer Thanks for your answers!