ADD new surrogate benchmark for the exploring openml paper

[mlindauer]

Main problem:

module = importlib.import_module("hpolib.benchmarks.surrogates.exploring_openml")
benchmark = getattr(module, "XGBoost_3")(rng=seed)

Further issues:

• [x] meta_information is missing (in particular num_function_evals, name, f_opt (can be set to 0))
• [x] please remove the print statements

[mfeurer]

I just updated the code a bit, the output of:

import importlib

module = importlib.import_module("hpolib.benchmarks.surrogates.exploring_openml")
benchmark = getattr(module, "XGBoost_3")(rng=1)
print(benchmark)

is now:

/home/feurerm/miniconda/3-4.5.4/envs/smac3/bin/python3 /home/feurerm/sync_dir/projects/HPOlib2/HPOlib2/bug.py
/home/feurerm/miniconda/3-4.5.4/envs/smac3/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/home/feurerm/miniconda/3-4.5.4/envs/smac3/lib/python3.6/site-packages/lightgbm/basic.py:1209: UserWarning: categorical_feature in Dataset is overridden.
New categorical_feature is [1]
  'New categorical_feature is {}'.format(sorted(list(categorical_feature))))
<hpolib.benchmarks.surrogates.exploring_openml.XGBoost_3 object at 0x7fd6f7f8cda0>

Process finished with exit code 0

It takes a while (~5 minutes) to build the surrogate, though.

[mlindauer]

Now, it takes really a lot of time. I'm trying to import XGBoost_3 now for roughly half an hour. Unfortunately, there is not output and hence, I don't know what it is doing at all. Could you please some (sparse) output such that a user does not believe that he/she hits an infinite loop?

Is the reproducibility still ensured since the HPO runs every time again? So, can I validate the configuration I found on my local machine again and it will return the same results?

[mfeurer]

Okay @KEggensperger I think I addressed all your requests.