search

automl / HPOlib1.5

GNU General Public License v3.0
69 stars 27 forks source link

Update Auto-sklearn benchmark #62

Closed mfeurer closed 5 years ago

KEggensperger commented 5 years ago

@mfeurer This is not yet ready, correct? Could you ping me once you think this is ready to be merged?

codecov-io commented 5 years ago

Codecov Report

:exclamation: No coverage uploaded for pull request base (development@ae33535). Click here to learn what that means. The diff coverage is 69.38%.

Impacted file tree graph

@@              Coverage Diff               @@
##             development      #62   +/-   ##
==============================================
  Coverage               ?   54.01%           
==============================================
  Files                  ?       38           
  Lines                  ?     2394           
  Branches               ?        0           
==============================================
  Hits                   ?     1293           
  Misses                 ?     1101           
  Partials               ?        0
Impacted Files Coverage Δ
hpolib/benchmarks/ml/sklearn_mlp_benchmark.py 50.61% <ø> (ø)
hpolib/benchmarks/ml/autosklearn_benchmark.py 81.81% <69.38%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update ae33535...0393cff. Read the comment docs.

KEggensperger commented 5 years ago

@mfeurer Can you have a look at the error message on Travis? If this is solved, we can finally merge ...

ERROR: test_load_openml (test_openml_data_manager.TestDataManager)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/automl/HPOlib2/tests/test_openml_data_manager.py", line 10, in test_load_openml
    X_train, y_train, X_val, y_val, X_test, y_test = dm.load()
  File "/home/travis/build/automl/HPOlib2/hpolib/util/openml_data_manager.py", line 94, in load
    self.task_id)
  File "/home/travis/build/automl/HPOlib2/hpolib/util/openml_data_manager.py", line 14, in _load_data
    task = openml.tasks.get_task(task_id)
  File "/home/travis/miniconda/envs/testenv/lib/python3.5/site-packages/openml/tasks/functions.py", line 310, in get_task
    raise e
  File "/home/travis/miniconda/envs/testenv/lib/python3.5/site-packages/openml/tasks/functions.py", line 301, in get_task
    task.download_split()
  File "/home/travis/miniconda/envs/testenv/lib/python3.5/site-packages/openml/tasks/task.py", line 116, in download_split
    split = OpenMLSplit._from_arff_file(cached_split_file)
  File "/home/travis/miniconda/envs/testenv/lib/python3.5/site-packages/openml/tasks/split.py", line 90, in _from_arff_file
    type_idx = meta._attrnames.index('type')
AttributeError: 'MetaData' object has no attribute '_attrnames'
mfeurer commented 5 years ago

That's actually an issue with OpenML at the moment, not sure what happened there. I can check this after Friday.