HIPS / Spearmint

Spearmint Bayesian optimization codebase
Other
1.54k stars 328 forks source link

Hanging after "Using database at localhost." #25

Open mechaman opened 9 years ago

mechaman commented 9 years ago

After I used ctr-c in the middle of a job, I restarted spearmint to continue. Unfortunately, spearmint gets hung up after "Using database at localhost.". With a little debugging, I noticed that it is not entering the while loop --- while resource.acceptingJobs(jobs): ---.

Is this due to MongoDB being hung up on a particular job? Some insight would be much appreciated :)

mechaman commented 9 years ago

Fixed! Mongodb was hung up on a particular job that had an error so that job was constantly pending. This error was in the file where my function definition resided. To start new and get rid of the previous state that was pending, I had to get rid of the spearmint.0 and spearmint.ns file in /data/db. Also, kill the mongod process and start a new one. After executing Spearmint's main.py again, the script started executing from job 1 again.

mikowals commented 8 years ago

I encountered this also. The fix for me was to remove the pending jobs from mongo. So for an experiment named 'exp_1' the commands were:

mongo
use spearmint
db['exp_1.jobs'].remove({status:'pending'})
cmleecm commented 8 years ago

I am having the same problem when I run /example/simple.

mechaman: Where do I find spearmint.0 and spearmint.ns? Is /data/db the data directory specified when forking mongod?

mikowals: For my case, I should do mongo use spearmint db['simple-braninhoo-example.jobs'].remove({status:'pending'}) ?