be-marc
commented
2 years ago Hey, maybe sections 3.2.4 Multifidelity and Hyperband of our paper Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges gives you a better understanding.
In summary, the multifidelity concepts assumes that the budget parameter influences the computational cost of the fitting the models and more budget is better in terms of predictive performance. Both are not true for max.depth and mtry. For random forest, you can use subsampling (see readme example) or num.trees.
Hyperband is a good approach if the evaluation of hyperparameter configurations is expensive and the search space is rather large. If you tune a ranger model with two hyperparameters on a task with 1000 observations, hyperband will not have an advantage over other methods.
MislavSag
Hi,
My questoin id probbably more suitable for some other thread, but I will try here :)
How do you in general choose budget parameter? Does the badget parameter influence significantly on the results of the optimization?
For example, for ranger learner I ussualy tune 2 hyperparamteres: max.depth and mtry. How do I know which is better as budget parameter?
And even more general, is hyperband in general better approach than other tuning mehods available in mlr3tuning? Does, in your experience, the choice of tuning method influnce the final result? It seems that in most cases the results are very similiar.