search

automl / RoBO

RoBO: a Robust Bayesian Optimization framework
BSD 3-Clause "New" or "Revised" License
481 stars 133 forks source link

Example computation time #40

Closed mglowacki100 closed 7 years ago

mglowacki100 commented 7 years ago

How long it should take to calculate example: https://github.com/automl/RoBO/blob/master/examples/example_fmin_fabolas.py I've changed num_iterations to 1 and after more than 10 minutes no results. One CPU core is on 100%.

Maybe it is related to warnings: 1. Using gpu device 0: GeForce GTX 980M (CNMeM is enabled with initial size: 90.0% of memory, cuDNN Mixed dnn version. The header is from one version, but we link with a different version (4007, 5103)) But theano GPU tests seems to work correctly.

2. /usr/local/lib/python2.7/dist-packages/numpy/lib/shape_base.py:873: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future return c.reshape(shape_out)

Btw1. I've tried to run RoBO on python3.5 but I have problems with george library, namely pip3 seems to install it, but there is no george in "3.5 ... -packages" and python can't import it.

Btw2. Have you compared RoBO with SigOpt.com ?

aaronkl commented 7 years ago

Hi, setting num_iterations=1 only specifies the number of iterations for the BO solver without the initial design (which is by default n_init=40 point). That means you still evaluate the 40 points of the initial design. The first warning was an deprecated import which should be fixed now.

For Fabolas you have to use our fork of george (https://github.com/sfalkner/george), because of our new kernels that we implemented. A simple: for req in $(cat requirements.txt); do pip install $req; done inside the RoBO directory should do the job. Let me know if this solved your problem.

No we haven't compared it to SigOpt yet but judging from their workshop paper it seems to be quite similar to vanilla Bayesian Optimization

cheers Aaron

mglowacki100 commented 7 years ago

Thanks for reply. In requirements.txt, there is numpy >= 1.7, there should be numpy>=1.7 - to remove parse error. I have to add sudo before python setup.py install, but it shouldn't be an impact.. Let's let it run for a couple of hours. I think that "problem" could be in SVC - it uses only 1 core, and from my expierence svm's rather slow.

mglowacki100 commented 7 years ago

It works (c.a. 2 hours on Intel® Core™ i7-6820HK CPU @ 2.70GHz × 8 , GeForce GTX 980M/PCIe/SSE2, 64GB ram)