Error while running run_magic_imputation

[aditisk]

Hello,

I am trying to run Palantir on my dataset to be able to eventually use Cellrank2. I am getting an error message (pasted below). I have loaded the scipy module and imported the necessary features as mentioned in the tutorial. How can I fix this? Thanks.

`--------------------------------------------------------------------------- _RemoteTraceback Traceback (most recent call last) _RemoteTraceback: """ Traceback (most recent call last): File "/ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 616, in wait_result_broken_or_wakeup result_item = result_reader.recv() File "/ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/multiprocessing/connection.py", line 251, in recv return _ForkingPickler.loads(buf.getbuffer()) ModuleNotFoundError: No module named 'scipy.sparse._csr' """

The above exception was the direct cause of the following exception:

BrokenProcessPool Traceback (most recent call last)

in ----> 1 imputed_X = palantir.utils.run_magic_imputation(adata) ~/.local/lib/python3.8/site-packages/palantir/utils.py in run_magic_imputation(data, dm_res, n_steps, sim_key, expression_key, imputation_key, n_jobs) 570 571 # Run the dot product in parallel on chunks --> 572 res = Parallel(n_jobs=n_jobs)( 573 delayed(_dot_helper_func)(T_steps, X[:, chunks[i - 1] : chunks[i]]) 574 for i in range(1, len(chunks)) /ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/site-packages/joblib/parallel.py in __call__(self, iterable) 1052 1053 with self._backend.retrieval_context(): -> 1054 self.retrieve() 1055 # Make sure that we get a last message telling us we are done 1056 elapsed_time = time.time() - self._start_time /ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/site-packages/joblib/parallel.py in retrieve(self) 931 try: 932 if getattr(self._backend, 'supports_timeout', False): --> 933 self._output.extend(job.get(timeout=self.timeout)) 934 else: 935 self._output.extend(job.get()) /ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout) 540 AsyncResults.get from multiprocessing.""" 541 try: --> 542 return future.result(timeout=timeout) 543 except CfTimeoutError as e: 544 raise TimeoutError from e /ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/concurrent/futures/_base.py in result(self, timeout) 437 raise CancelledError() 438 elif self._state == FINISHED: --> 439 return self.__get_result() 440 else: 441 raise TimeoutError() /ihome/crc/install/python/ondemand-jupyter-python3.8/lib/python3.8/concurrent/futures/_base.py in __get_result(self) 386 def __get_result(self): 387 if self._exception: --> 388 raise self._exception 389 else: 390 return self._result BrokenProcessPool: A result has failed to un-serialize. Please ensure that the objects returned by the function are always picklable.`

[ManuSetty]

This appears to be an issue with versioning of different python modules. Can you please try again with a new conda environment? We also suggest using a python version > 3.8

[aditisk]

Thanks for the suggestions. I tried using v3.9 and v3.10 in a virtual environment but now I am unable to run PCA (and all steps after that) on my adata object.

`--------------------------------------------------------------------------- ValueError Traceback (most recent call last) Cell In[10], line 2 1 # Note in the manuscript, we did not use highly variable genes but scanpy by default uses only highly variable genes ----> 2 sc.tl.pca(adata)

File ~/.local/lib/python3.10/site-packages/scanpy/preprocessing/_pca.py:200, in pca(data, n_comps, zero_center, svd_solver, random_state, return_info, use_highly_variable, dtype, copy, chunked, chunk_size) 194 if svd_solver not in {'lobpcg', 'arpack'}: 195 raise ValueError( 196 'svd_solver: {svd_solver} can not be used with sparse input.\n' 197 'Use "arpack" (the default) or "lobpcg" instead.' 198 ) --> 200 output = _pca_with_sparse( 201 X, n_comps, solver=svd_solver, random_state=random_state 202 ) 203 # this is just a wrapper for the results 204 X_pca = output['X_pca']

File ~/.local/lib/python3.10/site-packages/scanpy/preprocessing/_pca.py:303, in _pca_with_sparse(X, npcs, solver, mu, random_state) 292 return XHmat(x) - mhmat(ones(x)) 294 XL = LinearOperator( 295 matvec=matvec, 296 dtype=X.dtype, (...) 300 rmatmat=rmatmat, 301 ) --> 303 u, s, v = svds(XL, solver=solver, k=npcs, v0=random_init) 304 u, v = svd_flip(u, v) 305 idx = np.argsort(-s)

File ~/.local/lib/python3.10/site-packages/scipy/sparse/linalg/_eigen/_svds.py:443, in svds(A, k, ncv, tol, which, v0, maxiter, return_singular_vectors, solver, random_state, options) 104 def svds(A, k=6, ncv=None, tol=0, which='LM', v0=None, 105 maxiter=None, return_singular_vectors=True, 106 solver='arpack', random_state=None, options=None): 107 """ 108 Partial singular value decomposition of a sparse matrix. 109 (...) 441 442 """ --> 443 args = _iv(A, k, ncv, tol, which, v0, maxiter, return_singular_vectors, 444 solver, random_state) 445 (A, k, ncv, tol, which, v0, maxiter, 446 return_singular_vectors, solver, random_state) = args 448 largest = (which == 'LM')

File ~/.local/lib/python3.10/site-packages/scipy/sparse/linalg/_eigen/_svds.py:40, in _iv(A, k, ncv, tol, which, v0, maxiter, return_singular, solver, random_state) 37 if not (np.issubdtype(A.dtype, np.complexfloating) 38 or np.issubdtype(A.dtype, np.floating)): 39 message = "A must be of floating or complex floating data type." ---> 40 raise ValueError(message) 41 if np.prod(A.shape) == 0: 42 message = "A must not be empty."

ValueError: A must be of floating or complex floating data type.`