Hello, I'm running very large samples with lots of genes (> 30,000) and before it gave me "Job killed" but then I moved to a bigger cluster and it gives me:
Maximum number of Gibbs sampling iterations: 1000; terminating Gibbs sampling now.
Optimizing parameters for optimal clusters.
Cluster 1, 3557 genes
Traceback (most recent call last):
File "DP_GP_cluster.py", line 623, in
iter_num_at_birth=iter_num)
File "DP_GP/core.pyx", line 233, in DP_GP.core.dp_cluster.init (DP_GP/core.c:6979)
File "/py27/lib/python2.7/site-packages/GPy/core/parameterization/parameterized.py", line 27, in call
self.parameters_changed()
File "/py27/lib/python2.7/site-packages/GPy/core/gp.py", line 188, in parameters_changed
self.posterior, self._log_marginal_likelihood, self.grad_dict = self.inference_method.inference(self.kern, self.X, self.likelihood, self.Y_normalized, self.mean_function, self.Y_metadata)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/inference/latent_function_inference/exact_gaussian_inference.py", line 52, in inference
K = kern.K(X)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/kernel_slice_operations.py", line 79, in wrap
ret = f(self, s.X, s.X2, *a, kw)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 184, in call
return cacher(*args, *kwargs)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 120, in call
self.add_to_cache(cache_id, inputs, self.operation(args, kw))
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/add.py", line 39, in K
return reduce(np.add, (p.K(X, X2) for p in which_parts))
MemoryError
Do I need more CPUs or something? Or is it really not possible to cluster samples with so many genes?
Thanks!
Hello, I'm running very large samples with lots of genes (> 30,000) and before it gave me "Job killed" but then I moved to a bigger cluster and it gives me: Maximum number of Gibbs sampling iterations: 1000; terminating Gibbs sampling now. Optimizing parameters for optimal clusters. Cluster 1, 3557 genes Traceback (most recent call last): File "DP_GP_cluster.py", line 623, in
iter_num_at_birth=iter_num)
File "DP_GP/core.pyx", line 233, in DP_GP.core.dp_cluster.init (DP_GP/core.c:6979)
File "/py27/lib/python2.7/site-packages/GPy/core/parameterization/parameterized.py", line 27, in call
self.parameters_changed()
File "/py27/lib/python2.7/site-packages/GPy/core/gp.py", line 188, in parameters_changed
self.posterior, self._log_marginal_likelihood, self.grad_dict = self.inference_method.inference(self.kern, self.X, self.likelihood, self.Y_normalized, self.mean_function, self.Y_metadata)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/inference/latent_function_inference/exact_gaussian_inference.py", line 52, in inference
K = kern.K(X)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/kernel_slice_operations.py", line 79, in wrap
ret = f(self, s.X, s.X2, *a, kw)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 184, in call
return cacher(*args, *kwargs)
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/util/caching.py", line 120, in call
self.add_to_cache(cache_id, inputs, self.operation(args, kw))
File "/.conda/envs/py27/lib/python2.7/site-packages/GPy/kern/_src/add.py", line 39, in K
return reduce(np.add, (p.K(X, X2) for p in which_parts))
MemoryError
Do I need more CPUs or something? Or is it really not possible to cluster samples with so many genes? Thanks!