SeppeDeWinter
commented
1 year ago Hi @YayanFeng55
I would suggest to use the code in the development branch. See this discussion on how to run it: https://github.com/aertslab/scenicplus/discussions/202
All the best,
Seppe
Dear Sepp,
I was running SCENIC+ on linux system. It works well with other steps. But when I run run_scenicplus with below script,
Running SCENIC+
from scenicplus.wrappers.run_scenicplus import run_scenicplus run_scenicplus( scplus_obj = scplus_obj, variable = ['GEX_Cluster'], species = 'hsapiens', assembly = 'hg38', tf_file = '/hdd/yayan/Softwares/scenicplus/resources/TF_names_v_1.01.txt', save_path = os.path.join(outDir, 'scenicplus'), biomart_host = biomart_host, upstream = [1000, 150000], downstream = [1000, 150000], calculate_TF_eGRN_correlation = True, calculate_DEGs_DARs = True, export_to_loom_file = True, export_to_UCSC_file = True, path_bedToBigBed = '/hdd/yayan/Softwares', n_cpu = 25, _temp_dir = os.path.join("/hdd/yayan/", 'tmp'))
I got below error:
2023-10-27 08:41:53,508 SCENIC+_wrapper INFO Calculating gene ranking 2023-10-27 08:48:24,671 SCENIC+_wrapper INFO Calculating eGRNs gene based AUC 2023-10-27 08:50:35,358 SCENIC+_wrapper INFO Calculating TF-eGRNs AUC correlation 2023-10-27 08:57:02,697 SCENIC+_wrapper INFO Binarizing eGRNs AUC OpenBLAS blas_thread_init: pthread_create failed for thread 3 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 4 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 5 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 6 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 7 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 8 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 9 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 10 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 11 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 12 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 13 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 14 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 15 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 16 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 17 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 18 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 19 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 20 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 21 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 22 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 23 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 24 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 25 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 26 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 27 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 28 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 29 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 30 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 31 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 32 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 33 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 34 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 35 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 36 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 37 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 38 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 39 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 40 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 41 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 42 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 43 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 44 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 45 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 46 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 47 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 48 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 49 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 50 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 51 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 52 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 53 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 54 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 55 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 56 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 57 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 58 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 59 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 60 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 61 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 62 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max OpenBLAS blas_thread_init: pthread_create failed for thread 63 of 64: Resource temporarily unavailable OpenBLAS blas_thread_init: RLIMIT_NPROC 4096 current, 6185313 max Process ForkPoolWorker-60: Traceback (most recent call last): File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, self._kwargs) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, *kwds)) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/multiprocessing/pool.py", line 51, in starmapstar return list(itertools.starmap(args[0], args[1])) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/pyscenic/binarization.py", line 62, in derive_threshold gmm2 = mixture.GaussianMixture( File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/mixture/_base.py", line 181, in fit self.fit_predict(X, y) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/base.py", line 1152, in wrapper return fit_method(estimator, args, kwargs) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/mixture/_base.py", line 235, in fit_predict self._initialize_parameters(X, random_state) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/mixture/_base.py", line 115, in _initialize_parameters cluster.KMeans( File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/base.py", line 1152, in wrapper return fit_method(estimator, *args, **kwargs) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py", line 1530, in fit labels, inertia, centers, niter = kmeans_single( File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/cluster/_kmeans.py", line 688, in _kmeans_single_lloyd with threadpool_limits(limits=1, user_api="blas"): File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/sklearn/utils/fixes.py", line 72, in threadpool_limits return controller.limit(limits=limits, user_api=user_api) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/threadpoolctl.py", line 769, in limit return _ThreadpoolLimiter(self, limits=limits, user_api=user_api) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/threadpoolctl.py", line 435, in init self._set_threadpool_limits() File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/threadpoolctl.py", line 568, in _set_threadpool_limits lib_controller.set_num_threads(num_threads) File "/hdd/yayan/Softwares/miniconda3/envs/scenicplus/lib/python3.8/site-packages/threadpoolctl.py", line 185, in set_num_threads return set_func(num_threads) KeyboardInterrupt Numba: Attempted to fork from a non-main thread, the TBB library may be in an invalid state in the child process.
Version (please complete the following information):
Additional context Now I have successfully obtained region_ranking.pkl and gene_ranking.pkl. Because the assay is quite large, I usually spent 5 days to run this code. How should I fix above error ? If I rerun above code, could I use the resulting files (region_ranking.pkl + gene_ranking.pkl)?
All the best, Yayan