AIworx-Labs / chocolate

A fully decentralized hyperparameter optimization framework
http://chocolate.readthedocs.io
BSD 3-Clause "New" or "Revised" License
120 stars 41 forks source link

Error in "Optimizing Over Multiple Models" example #18

Closed Codeguyross closed 5 years ago

Codeguyross commented 5 years ago

The score_svm example on the Optimizing Over Multiple Models has the following function definition:

def score_svm(X, y, algo, **params):

When the score_svm function is called, it is called without the algo parameter (see below). What is actually supposed to be sent for the algo parameter? loss = score_svm(X, y, **params)

fmder commented 5 years ago

When calling, the algo is in the params. I agree that the different signature can be confusing.

Codeguyross commented 5 years ago

Im sorry, I dont quite understand. It doesnt run without algo in the params and throws an error. What is suppose to be in the parameter algo?

This is the score_svm function which uses algo:

def score_svm(X, y, algo, **params):
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

    clf = algo(**params)
    clf.fit(X_train, y_train)
    y_pred = clf.predict(X_test)

    return -f1_score(y_test, y_pred)

This is the error that is generated from trying to call score_svm without algo:

TypeError                                 Traceback (most recent call last)
<ipython-input-8-4f966665fab3> in <module>()
      6 
      7 token, params = sampler.next()
----> 8 loss = score_svm(X, y, **params)
      9 sampler.update(token, loss)

TypeError: score_svm() missing 1 required positional argument: 'algo'
fmder commented 5 years ago

algo is a parameter returned by next(). The returned dictionary should contain a key "algo".

Codeguyross commented 5 years ago

Ok, were getting somewhere but its not quite working yet

from sklearn.datasets import make_classification
X, y = make_classification(n_samples=80000, random_state=1)
conn = choco.SQLiteConnection(url="sqlite:///db.db")
sampler = choco.QuasiRandom(conn, space, seed=42, skip=0, clear_db=True)
token, params = sampler.next()
params
{'C': 100.0,
 'algo': sklearn.svm.classes.SVC,
 'gamma': 6.309573444801943e-05,
 'kernel': 'rbf'}
params['algo']
sklearn.svm.classes.SVC
algo= params['algo']
loss = score_svm(X, y, algo, **params)
sampler.update(token, loss)

Produces this error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-2b6ad5188828> in <module>()
      1 algo= params['algo']
      2 
----> 3 loss = score_svm(X, y, algo, **params)
      4 sampler.update(token, loss)

TypeError: score_svm() got multiple values for argument 'algo'

Thanks for the help!

P.S. If it helps, that error is produced from trying either of these two space variables taken from the example:

space = [{"algo" : SVC, "kernel" : "rbf",
              "C" : choco.log(low=-2, high=10, base=10),
              "gamma" : choco.log(low=-9, high=3, base=10)},
         {"algo" : SVC, "kernel" : "poly",
              "C" : choco.log(low=-2, high=10, base=10),
              "gamma" : choco.log(low=-9, high=3, base=10),
              "degree" : choco.quantized_uniform(low=1, high=5, step=1),
              "coef0" : choco.uniform(low=-1, high=1)},
         {"algo" : LinearSVC,
              "C" : choco.log(low=-2, high=10, base=10),
              "penalty" : choco.choice(["l1", "l2"])}]

space = {"algo" : {SVC : {"gamma" : choco.log(low=-9, high=3, base=10)},
                          "kernel" : {"rbf" : None,
                                      "poly" : {"degree" : choco.quantized_uniform(low=1, high=5, step=1),
                                                "coef0" : choco.uniform(low=-1, high=1)}},
                   LinearSVC : {"penalty" : choco.choice(["l1", "l2"])}},
         "C" : choco.log(low=-2, high=10, base=10)}
fmder commented 5 years ago

Yes because it is already in the params dictionary. What is your version of Python?

Le jeu. 13 déc. 2018 13 h 47, Codeguyross notifications@github.com a écrit :

Ok, were getting somewhere but its not quite working yet

from sklearn.datasets import make_classification X, y = make_classification(n_samples=80000, random_state=1) conn = choco.SQLiteConnection(url="sqlite:///db.db") sampler = choco.QuasiRandom(conn, space, seed=42, skip=0, clear_db=True) token, params = sampler.next()

params {'C': 100.0, 'algo': sklearn.svm.classes.SVC, 'gamma': 6.309573444801943e-05, 'kernel': 'rbf'}

params['algo'] sklearn.svm.classes.SVC

algo= params['algo'] loss = score_svm(X, y, algo, **params) sampler.update(token, loss)

Produces this error


TypeError Traceback (most recent call last)

in () 1 algo= params['algo'] 2 ----> 3 loss = score_svm(X, y, algo, **params) 4 sampler.update(token, loss) TypeError: score_svm() got multiple values for argument 'algo' Thanks for the help! — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub , or mute the thread .