Closed Mehrian closed 9 years ago
mgelbart
commented
9 years ago Hi, What command exactly are you typing at the last step? It sounds like you might just be typing "python main.py". You need to give it the path to the experiment directory, like "python main,py ../examples/distributed/" or "python main.py Desktop/Spearmint-master/examples/distributed/". This should work.
Mike
Mehrian
commented
9 years ago Hi Mike Thanks! This command worked! "python main,py ../examples/distributed/"
Now, it starts running and the following error is repeatedly showed!
/bin/sh: 1: sbatch: not found Failed to submit job 1267.
Resources: NAME PENDING COMPLETE
my-machine 0 1
cluster 0 0
_TOTAL_ 0 1 Getting suggestion...
Suggestion: NAME TYPE VALUE
y float 7.500000
x float 2.500000 /bin/sh: 1: sbatch: not found Failed to submit job 1268.
Resources: NAME PENDING COMPLETE
my-machine 0 1
cluster 0 0
_TOTAL_ 0 1 I'm wondering if I have to sth elase! Best Mohammad
mgelbart
commented
9 years ago Try the "simple" example rather than the "distributed" one: "python main,py ../examples/simple/"
Mehrian
commented
9 years ago Hi Mike I ran the simple branin and there is no problem executing....now the output is saved in a folder (output). I'm wondering if there is any stop criteria for the code? because I terminate the process manually while the last output file was 00000171.out Now, I don't know how to interpret the output (plotting figures and...) Also, would you please let me know if there is any tutorial for spearmint package?
Another issue is that, it seems that using spearmint package is time consuming!! because I ran the branin example with a GA(Genetic Algorithm) code, and it took 0.1 second to give me the best result!! While I want to use Bayesian Optimization for a model that is really time consuming(each sampling takes 6 hours to calculate!!) and the samples should be selected very carefully! (GA is not an appropriate approach for my case!) I'm wondering if you think Bayesian optimization (spearmint) could be helpful for my case?
JasperSnoek
commented
9 years ago Hi Mehrian, yes Bayesian optimization is exactly well suited to problems that are more "expensive". Try plotting Braninhoo vs. GA as a function of evaluations of the Braninhoo function. You'll see that the Bayesian optimization requires an order of magnitude less function evaluations. The Bayesian optimization procedure involves building a statistical model of the global objective, whereas genetic algorithms use a simple heuristic based on local information to suggest new experiments close to the best seen so far. The whole idea of Bayesian optimization is that when an experiment takes any real amount of time (or resources), it really pays off to spend a little more time carefully modeling the space and reasoning about the next experiment to run.
On Mon, May 4, 2015 at 5:49 AM, Mehrian notifications@github.com wrote:
Hi Mike I ran the simple branin and there is no problem executing....now the output is saved in a folder (output). I'm wondering if there is any stop criteria for the code? because I terminate the process manually while the last output file was 00000171.out Now, I don't know how to interpret the output (plotting figures and...) Also, would you please let me know if there is any tutorial for spearmint package?
Another issue is that, it seems that using spearmint package is time consuming!! because I ran the branin example with a GA(Genetic Algorithm) code, and it took 0.1 second to give me the best result!! While I want to use Bayesian Optimization for a model that is really time consuming(each sampling takes 6 hours to calculate!!) and the samples should be selected very carefully! (GA is not an appropriate approach for my case!) I'm wondering if you think Bayesian optimization (spearmint) could be helpful for my case?
— Reply to this email directly or view it on GitHub https://github.com/HIPS/Spearmint/issues/18#issuecomment-98654836.
Mehrian
commented
9 years ago Hi Jasper Thanks for your information on Bayesian optimization. The only problem with Spearmint package is that, there is no tutorial on that!!! Now, I'm trying to create a callable objective function and as it was mentioned I created the config.json file and add another parameter(z) to it(I have 3 par). Also, I changed the file like below
"language" : "PYTHON",
"main-file" : "Neo.py", ---------> the name of my function!
"experiment-name" : "simple-braninhoo-example", ???????
"likelihood" : "NOISELESS",??????Now, when I run the main.py, I have the following error!
Traceback (most recent call last):
File "main.py", line 494, in
Would you please let me know if there is any tutorial or suggest me a way to become more familiar with this package! Best wishes, Mohammad
JasperSnoek
commented
9 years ago Hi Mohammad, I think the problem here is that you have to clean up your
experiment before adding another parameter. In the spearmint directory
type 'cleanup.sh
Jasper
On Mon, May 4, 2015 at 10:42 AM, Mehrian notifications@github.com wrote:
Hi Jasper Thanks for your information on Bayesian optimization. The only problem with Spearmint package is that, there is no tutorial on that!!! Now, I'm trying to create a callable objective function and as it was mentioned I created the config.json file and add another parameter(z) to it(I have 3 par). Also, I changed the file like below
"language" : "PYTHON", "main-file" : "Neo.py", ---------> the name of my function! "experiment-name" : "simple-braninhoo-example", ??????? "likelihood" : "NOISELESS",??????
Now, when I run the main.py, I have the following error!
Traceback (most recent call last): File "main.py", line 494, in main() File "main.py", line 286, in main suggested_job = get_suggestion(chooser, resource.tasks, db, expt_dir, options, resource_name) File "main.py", line 355, in get_suggestion hypers = chooser.fit(task_group, hypers, task_options) File "/home/mohammad/Desktop/Spearmint-master/spearmint/choosers/default_chooser.py", line 309, in fit hypers=hypers.get(task_name, None) File "/home/mohammad/Desktop/Spearmint-master/spearmint/models/gp.py", line 505, in fit self._hypers_list = self._collect_samples(self.mcmc_iters) File "/home/mohammad/Desktop/Spearmint-master/spearmint/models/gp.py", line 378, in _collect_samples sampler.sample(self) File "/home/mohammad/Desktop/Spearmint-master/spearmint/sampling/slice_sampler.py", line 262, in sample params_array, current_ll = slice_sample(params_array, self.logprob, model, *
_self.sampler_options) File "/home/mohammad/Desktop/Spearmint-master/spearmint/sampling/mcmc.py", line 354, in slice_sample new_x, new_llh = direction_slice(direction, init_x) File "/home/mohammad/Desktop/Spearmint-master/spearmint/sampling/mcmc.py", line 289, in direction_slice llh_s = np.log(npr.rand()) + dir_logprob(0.0) File "/home/mohammad/Desktop/Spearmint-master/spearmint/sampling/mcmc.py", line 272, in dir_logprob return logprob(direction_z + init_x, logprob_args) File "/home/mohammad/Desktop/Spearmint-master/spearmint/sampling/slice_sampler.py", line 240, in logprob lp += model.log_likelihood() File "/home/mohammad/Desktop/Spearmint-master/spearmint/models/gp.py", line 536, in log_likelihood cov = self.kernel.cov(self.observed_inputs) File "/home/mohammad/Desktop/Spearmint-master/spearmint/kernels/transform_kernel.py", line 200, in cov return self.kernel.cov(self.transformer.forward_pass(inputs)) File "/home/mohammad/Desktop/Spearmint-master/spearmint/transformations/transformer.py", line 262, in forward_pass layer_out[:,i:i+t_len] = transformation.forward_pass(prev_layer[:,inds]) File "/home/mohammad/Desktop/Spearmint-master/spearmint/transformations/beta_warp.py", line 212, in inner return func(cls_instance, inputs, args) File "/home/mohammad/Desktop/Spearmint-master/spearmint/transformations/beta_warp.py", line 245, in forward_pass return sps.beta.cdf(inputs, self.alpha.value, self.beta.value) File "/home/mohammad/anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.py", line 1648, in cdf cond2 = (x >= self.b) & cond0 ValueError: operands could not be broadcast together with shapes (422,3) (2,)
Would you please let me know if there is any tutorial or suggest me a way to become more familiar with this package! Best wishes, Mohammad
— Reply to this email directly or view it on GitHub https://github.com/HIPS/Spearmint/issues/18#issuecomment-98735905.
Mehrian
commented
9 years ago Hi Jasper I tried that command, but the result is : cleanup.sh: command not found
I think the problem is with the "experiment name" in the config file! because when I try to run my code, the error is as follows:
Traceback (most recent call last):
File "main.py", line 495, in
So, would you please let me know what is experiment name??? Best regards, Mohammad
JasperSnoek
commented
9 years ago Try either ./cleanup.sh or "bash cleanup.sh".
Take a look at the example config files (e.g. https://github.com/HIPS/Spearmint/blob/master/examples/simple/config.json). Just change the value of the variable "experiment-name".
Jasper
On Wed, May 6, 2015 at 5:20 PM, Mehrian notifications@github.com wrote:
Hi Jasper I tried that command, but the result is : cleanup.sh: command not found
I think the problem is with the "experiment name" in the config file! because when I try to run my code, the error is as follows:
Traceback (most recent call last): File "main.py", line 495, in main() File "main.py", line 287, in main suggested_job = get_suggestion(chooser, resource.tasks, db, expt_dir, options, resource_name) File "main.py", line 342, in get_suggestion experiment_name = options['experiment-name'] KeyError: 'experiment-name'
So, would you please let me know what is experiment name??? Best regards, Mohammad
— Reply to this email directly or view it on GitHub https://github.com/HIPS/Spearmint/issues/18#issuecomment-99613034.
Mehrian
commented
9 years ago Hi Jasper When I try that command for cleaning up in the spearmint directory, I have the following error:
bash: syntax error near unexpected token `newline'
Also, in my callable function(Neo.py), I have 3 parameters. I imported "pdb" to trace the function when it is running! But the problem is that, when I execute "main.py", it seems that, it doesn't calls my objective function. Because I put "pdb.set_trace()" in the script (Neo.py) to track it line by line, so it should stop at that specific line in my objective function, BUT it doesn't!!!!! the output is as follows (it constantly repeats!!)
*OUTPUT******* Getting suggestion...
Suggestion: NAME TYPE VALUE
y float 0.400000
x float 0.000100
z float 0.000010 Submitted job 33 with local scheduler (process id: 4579). Status: 1 pending, 0 complete.
############### My Objective Function(Neo.py) #######################
import numpy as np import math import os import pdb from pylab import *
dt = 3600*12 #time step in secondes tf = 2 #final time
pdb.set_trace() def Neo(x,y,z):
os.system("FreeFem++-mpi -ne -nw BioreactorGrowth.edp -regime Stokes -poro "\
+str(y)+" -PS "+str(z)+" -vin "+str(x)+" -DT "+str(dt)+" -TF "+str(tf)
myfile = open("NeoTissueVol.txt")
result=myfile.read()
print 'Result = %f' % result
#time.sleep(np.random.randint(60))
return resultdef main(job_id, params): print 'Anything printed here will end up in the output directory for job #%d' % job_id print params return Neo(params['x'], params['y'], params['z'])
##################################
Thanks in advance, Mohammad
Mehrian
commented
9 years ago Hi Jasper The problem with cleaning up is now OK !! I just have problem with my objective function(Neo.py) and as I said, it seems that it doesn't executes Neo.py and the output is always the same! and the written values for x, y and z doesn't change! Would you please help me.... Best wishes, Mohammad
Dear all I've installed the spearmint package and mongoDB on Linux. My problem is in the final step, where I want to run main.py for testing branin.py, first I go to this address ("Desktop/Spearmint-master/examples/distributed/") and execute the branin.py Then, in the same address I execute the command " mongod --fork --logpath --dbpath" everything is OK to this step! for running the main.py, first I go back to this address("Desktop/Spearmint-master/Spearmint") and when I execute the command I see the following error!
Traceback (most recent call last): File "main.py", line 494, in
main()
File "main.py", line 249, in main
options, expt_dir = get_options()
File "main.py", line 216, in get_options
expt_dir = os.path.realpath(os.path.expanduser(args[0]))
IndexError: list index out of range
I'm wondering where we have to define the branin parameters in the main.py ??? would you please help me to solve the error? Thank you in advance, Mohammad