Hi, I am wondering how many configurations are actually sampled during a run of Hyperband/BOHB. As explained in this issue, I understand that when, e.g., setting min_budget=1 and max_budget=10 with eta=3, Hyperband starts by selecting 10 configurations with budget=1.
Then from these, the best 1/eta (1/3 in this case) configs are taken to the next round and so on. This would mean if I n_iterations=1, only one SH run is performed and thus, 10 (first run) + 3 (second run) + 1 (last run) = 14 configurations should be executed overall, while only one with the largest budget.
However, when executing Hyperband for my example I get the following result:
on the result of Hyperband. I removed the exact configs with dummy values because I do not think they are important here. However, what I do not get, is why there are 3 configurations with the largest budget? Moreover, it happens that these are actually the same configurations. I tried it multiple time and the same happened, so it is not due to some non-deterministic behaviour.
In addition, why do I have to specify n_iterations, shouldn't that be determined by the budgets and eta? If I do not specify it, the default is 1, right?
Hi, I am wondering how many configurations are actually sampled during a run of Hyperband/BOHB. As explained in this issue, I understand that when, e.g., setting min_budget=1 and max_budget=10 with eta=3, Hyperband starts by selecting 10 configurations with budget=1. Then from these, the best 1/eta (1/3 in this case) configs are taken to the next round and so on. This would mean if I n_iterations=1, only one SH run is performed and thus, 10 (first run) + 3 (second run) + 1 (last run) = 14 configurations should be executed overall, while only one with the largest budget.
However, when executing Hyperband for my example I get the following result:
config budget loss 0 0 1.111111 1.065852 1 1 1.111111 0.804863 2 2 1.111111 0.709007 3 3 10.000000 0.391982 4 3 10.000000 0.337096 5 3 10.000000 0.337096 6 6 1.111111 0.575394 7 7 1.111111 0.550342 8 8 3.333333 0.467931 9 9 3.333333 0.400403 10 10 3.333333 0.468913 11 11 3.333333 0.461708 12 12 1.111111 1.027357
This is the result when calling
on the result of Hyperband. I removed the exact configs with dummy values because I do not think they are important here. However, what I do not get, is why there are 3 configurations with the largest budget? Moreover, it happens that these are actually the same configurations. I tried it multiple time and the same happened, so it is not due to some non-deterministic behaviour.
In addition, why do I have to specify
n_iterations, shouldn't that be determined by the budgets and eta? If I do not specify it, the default is 1, right?