search

slds-lmu / yahpo_gym

Surrogate benchmarks for HPO problems
26 stars 3 forks source link

Incorrect information from list_benchmarks #37

Closed sebffischer closed 2 years ago

sebffischer commented 2 years ago 
> yahpogym::list_benchmarks()
Key             Instances  Cat. HP    Cont. HP   Targets
lcbench         OpenML_task_id  1          8          6
fcnet           task            4          8          4
nb301           dataset         35         1          2
rbv2_svm        task_id         3          6          6
rbv2_ranger     task_id         4          7          6
rbv2_rpart      task_id         2          6          6
rbv2_glmnet     task_id         2          4          6
rbv2_xgboost    task_id         3          14         6
rbv2_aknn       task_id         3          6          6
rbv2_super      task_id         8          33         6
iaml_ranger     task_id         4          6          12
iaml_rpart      task_id         1          5          12
iaml_glmnet     task_id         1          3          12
iaml_xgboost    task_id         2          13         12
iaml_super      task_id         6          24         12

However LC Bench has these (7) hyperparameters (none of which is categorical) batch size, learning rate, max dropout, max units, momentum, num layers, weight decay

pfistfl commented 2 years ago

cat was an error, cont is 7 + 1 fideliy hp (epoch). should be clearer now.

pfistfl commented 2 years ago

A better overview can be obtained from here from now on: https://pfistfl.github.io/yahpo_gym/scenarios.html