search

HIPS / Spearmint

Spearmint Bayesian optimization codebase
Other
1.55k stars 328 forks source link

Is it possible to run Spearmint in some "manual" mode #82

Closed cuihenggang closed 8 years ago

cuihenggang commented 8 years ago

Hello,

I have been using the Spearmint tool from another codebase (JasperSnoek/spearmint), and I usually use the spearmint-lite interface there. With that interface, I just need to run spearmint-lite.py and it will write the proposed hyperparam choice to a result file. I will then manually run my machine learning application with that proposed hyperparam choice and report back the loss to the result file, and run spearmint-lite.py again to get another proposed hyperparam choice.

If I understand correctly, in this codebase, I can only talk with Spearmint with a callback python script (something like branin.py), and Spearmint will call branin.py to evaluate each hyperparam choice. Since it's very hard for me to launch my machine learning platform with a python script, and I would rather manually run each experiment, I'm wondering is there any way that I can manually run my experiments?

Thank you very much!

cuihenggang commented 8 years ago

Another way of asking this question: How can I call Spearmint, instead of having Spearmint call me?

Thank you very much!

cuihenggang commented 8 years ago

I have figured out how to do that. Thank you!

mgelbart commented 8 years ago

@cuihenggang: I was just in the middle of writing that we don't currently have this functionality. How did you approach it?

cuihenggang commented 8 years ago

I actually modified the main.py script to have it read the previous function values and call "chooser.suggest()" to get a new suggestion: https://github.com/cuihenggang/Spearmint-hips/blob/master/spearmint/mymain.py

I have got rid off all the task and task group stuffs though.

mgelbart commented 8 years ago

Makes sense. Thanks for sharing the link, others may find it useful.