search

automl / HPOBenchExperimentUtils

Experiment code to run large-scale experimente with HPOBench
Apache License 2.0
7 stars 5 forks source link

Rename hpobench #11

Closed NeoChaos12 closed 3 years ago

NeoChaos12 commented 3 years ago

In accordance with the renaming of HPOlib2 to HPOBench, all references to that repo have been renamed here. Also, this repo's own python module name has been changed to "HPOBenchExperimentUtils".

P.S.: To maintain conformity, the output files such as "hpolib_runhistory.txt" have also been renamed to "hpobench_runhistory.txt", so local scripts that are not a part of this repo might fail to run unless updated.

P.P.S.: I would appreciate a minimal testing of the changes, especially for BOHB and SMAC, by either @PhMueller or @KEggensperger just to ensure that nothing broke that is not visible to me.

KEggensperger commented 3 years ago

I added a few commits to fix the setup for running experiments. Also, @NeoChaos12 I allowed DEHB to run on all fidelity types.

NeoChaos12 commented 3 years ago

Seems good to me, although I would note here that normally sampled float/int parameters are not being handled when mapping from ConfigSpace to either emukit's ParameterSpace objects or dragonfly's internal representations. If I remember correctly, when we were discussing this issue there were no benchmarks that depended on normal sampling so we had deferred the implementation of these mappings to the future.

Is that future now?