JasperSnoek
commented
9 years ago Hi @mylesnovick, looks like some numerical instability in the Gaussian process code. Could you possibly share your observed inputs and values? My guess is that the observations are very noisy or oddly behaved, which makes modeling quite difficult.
On Mon, May 4, 2015 at 4:39 PM, mylesnovick notifications@github.com wrote:
After several successful runs, it seems spearmint crashes with the following output:
Fitting GP for main task... Traceback (most recent call last): File "main.py", line 494, in
main() File "main.py", line 286, in main suggested_job = get_suggestion(chooser, resource.tasks, db, expt_dir, options, resource_name) File "main.py", line 355, in get_suggestion hypers = chooser.fit(task_group, hypers, task_options) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/choosers/default_chooser.py", line 309, in fit hypers=hypers.get(task_name, None) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 505, in fit self._hypers_list = self._collect_samples(self.mcmc_iters) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 378, in _collect_samples sampler.sample(self) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/slice_sampler.py", line 262, in sample params_array, current_ll = slice_sample(params_array, self.logprob, model, _self.sampler_options) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 354, in slice_sample new_x, new_llh = direction_slice(direction, init_x) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 289, in direction_slice llh_s = np.log(npr.rand()) + dir_logprob(0.0) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 272, in dir_logprob return logprob(direction_z + init_x, logprob_args) File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/slice_sampler.py", line 240, in logprob lp += model.log_likelihood() File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 538, in log_likelihood solve = spla.cho_solve((chol, True), self.observed_values - self.mean.value) File "/usr/local/lib/python2.7/site-packages/scipy/linalg/decomp_cholesky.py", line 162, in cho_solve b1 = asarray_chkfinite(b) File "/usr/local/lib/python2.7/site-packages/numpy/lib/function_base.py", line 613, in asarray_chkfinite "array must not contain infs or NaNs") ValueError: array must not contain infs or NaNs
My config file is as follows, if helpful:
{ "language": "PYTHON", "main-file": "stub_optimizing.py", "experiment-name": "swingy-monkey-optimization", "likelihood": "GAUSSIAN", "variables": { "height_bucket_div": { "type": "FLOAT", "size": 1, "min": 1, "max": 15 }, "width_bucket_div": { "type": "FLOAT", "size": 1, "min": 1, "max": 10 }, "vel_bucket_len": { "type": "FLOAT", "size": 1, "min": 1, "max": 30 }, "discount": { "type": "FLOAT", "size": 1, "min": 0, "max": 1 }, "greed_exp": { "type": "FLOAT", "size": 1, "min": 1, "max": 3 } } }
— Reply to this email directly or view it on GitHub https://github.com/HIPS/Spearmint/issues/19.
mylesnovick
hanspinckaers
cwj006
After several successful runs, it seems spearmint crashes with the following output:
My config file is as follows, if helpful: