search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
187 stars 29 forks source link

Help with Kernal dead #259

Open Chrisdoan9 opened 1 year ago

Chrisdoan9 commented 1 year ago

Hi team,

Would you please suggest what cause the kernal dead error? Thank you so much! This cell ran for about 2 hours and then I got the error kernal dead. I put 1024Gb of memory and still got kernal dead.

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )

This is what I have in the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

I found a similar issue here: https://github.com/aertslab/SCENICprotocol/issues/28 I run Jupyter notebook on a server.

Version (please complete the following information):

SeppeDeWinter commented 1 year ago

Hi @Chrisdoan9

At what step is the kernel dying?

Also the development version of the code uses less memory, so you can also consider using that version.

See https://github.com/aertslab/scenicplus/discussions/202 for more info.

All the best,

Seppe

Chrisdoan9 commented 1 year ago

Hi @SeppeDeWinter

The kernel dying at this cell:

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )
SeppeDeWinter commented 1 year ago

Hi @Chrisdoan9

Did you have any output before the kernel died? And could you please copy-paste this? The function you pasted above will run many steps, so it's difficult for me to know what might be causing the issue.

Best,

Seppe

Chrisdoan9 commented 11 months ago

Hi @SeppeDeWinter, sure. That function printed a lot of output during about 2 hours of running:

2023-11-14 14:33:44,142 pycisTarget_wrapper INFO pbmc_tutorial/motifs folder already exists. 2023-11-14 14:33:45,422 pycisTarget_wrapper INFO Loading cisTarget database for topics_otsu 2023-11-14 14:33:45,423 cisTarget INFO Reading cisTarget database 2023-11-14 14:37:52,116 pycisTarget_wrapper INFO Running cisTarget for topics_otsu 2023-11-14 14:37:52,118 cisTarget INFO Running cisTarget for Topic1 which has 4811 regions 2023-11-14 14:38:15,144 cisTarget INFO Annotating motifs for Topic1 2023-11-14 14:38:18,547 cisTarget INFO Getting cistromes for Topic1 2023-11-14 14:38:18,939 cisTarget INFO Running cisTarget for Topic2 which has 7981 regions 2023-11-14 14:38:38,055 cisTarget INFO Annotating motifs for Topic2 2023-11-14 14:38:41,118 cisTarget INFO Getting cistromes for Topic2 2023-11-14 14:38:41,606 cisTarget INFO Running cisTarget for Topic3 which has 5233 regions 2023-11-14 14:39:01,134 cisTarget INFO Annotating motifs for Topic3 2023-11-14 14:39:04,537 cisTarget INFO Getting cistromes for Topic3 2023-11-14 14:39:05,020 cisTarget INFO Running cisTarget for Topic4 which has 3393 regions 2023-11-14 14:39:23,593 cisTarget INFO Annotating motifs for Topic4 2023-11-14 14:39:26,517 cisTarget INFO Getting cistromes for Topic4 2023-11-14 14:39:26,958 cisTarget INFO Running cisTarget for Topic5 which has 2608 regions 2023-11-14 14:39:45,569 cisTarget INFO Annotating motifs for Topic5 2023-11-14 14:39:49,003 cisTarget INFO Getting cistromes for Topic5 2023-11-14 14:39:49,659 cisTarget INFO Running cisTarget for Topic6 which has 1795 regions 2023-11-14 14:40:08,710 cisTarget INFO Annotating motifs for Topic6 2023-11-14 14:40:11,689 cisTarget INFO Getting cistromes for Topic6 2023-11-14 14:40:12,325 cisTarget INFO Running cisTarget for Topic7 which has 5383 regions 2023-11-14 14:40:31,390 cisTarget INFO Annotating motifs for Topic7 2023-11-14 14:40:34,348 cisTarget INFO Getting cistromes for Topic7 2023-11-14 14:40:34,863 cisTarget INFO Running cisTarget for Topic8 which has 6027 regions 2023-11-14 14:40:54,441 cisTarget INFO Annotating motifs for Topic8 2023-11-14 14:40:58,174 cisTarget INFO Getting cistromes for Topic8 2023-11-14 14:40:58,723 cisTarget INFO Running cisTarget for Topic9 which has 6579 regions 2023-11-14 14:41:18,911 cisTarget INFO Annotating motifs for Topic9 2023-11-14 14:41:22,620 cisTarget INFO Getting cistromes for Topic9 2023-11-14 14:41:23,350 cisTarget INFO Running cisTarget for Topic10 which has 7277 regions 2023-11-14 14:41:43,716 cisTarget INFO Annotating motifs for Topic10 2023-11-14 14:41:47,236 cisTarget INFO Getting cistromes for Topic10 2023-11-14 14:41:47,807 cisTarget INFO Running cisTarget for Topic11 which has 4394 regions 2023-11-14 14:42:06,965 cisTarget INFO Annotating motifs for Topic11 2023-11-14 14:42:10,277 cisTarget INFO Getting cistromes for Topic11 2023-11-14 14:42:10,882 cisTarget INFO Running cisTarget for Topic12 which has 8638 regions 2023-11-14 14:42:31,717 cisTarget INFO Annotating motifs for Topic12 2023-11-14 14:42:36,166 cisTarget INFO Getting cistromes for Topic12 2023-11-14 14:42:36,995 cisTarget INFO Running cisTarget for Topic13 which has 3038 regions 2023-11-14 14:42:56,722 cisTarget INFO Annotating motifs for Topic13 2023-11-14 14:43:00,047 cisTarget INFO Getting cistromes for Topic13 2023-11-14 14:43:00,567 cisTarget INFO Running cisTarget for Topic14 which has 6248 regions 2023-11-14 14:43:21,087 cisTarget INFO Annotating motifs for Topic14 2023-11-14 14:43:25,868 cisTarget INFO Getting cistromes for Topic14 2023-11-14 14:43:26,559 cisTarget INFO Running cisTarget for Topic15 which has 4824 regions 2023-11-14 14:43:46,414 cisTarget INFO Annotating motifs for Topic15 2023-11-14 14:43:49,907 cisTarget INFO Getting cistromes for Topic15 2023-11-14 14:43:50,422 cisTarget INFO Running cisTarget for Topic16 which has 9653 regions 2023-11-14 14:45:35,354 cisTarget INFO Getting cistromes for Topic3 2023-11-14 14:45:35,714 cisTarget INFO Running cisTarget for Topic4 which has 2426 regions 2023-11-14 14:45:53,764 cisTarget INFO Annotating motifs for Topic4 2023-11-14 14:45:56,499 cisTarget INFO Getting cistromes for Topic4 2023-11-14 14:45:56,838 cisTarget INFO Running cisTarget for Topic5 which has 277 regions 2023-11-14 14:46:15,899 cisTarget INFO Annotating motifs for Topic5 2023-11-14 14:46:18,438 cisTarget INFO Getting cistromes for Topic5 2023-11-14 14:46:18,892 cisTarget INFO Running cisTarget for Topic6 which has 512 regions 2023-11-14 14:46:38,179 cisTarget INFO Annotating motifs for Topic6 2023-11-14 14:46:40,905 cisTarget INFO Getting cistromes for Topic6 2023-11-14 14:46:41,348 cisTarget INFO Running cisTarget for Topic7 which has 3876 regions 2023-11-14 14:47:00,221 cisTarget INFO Annotating motifs for Topic7 2023-11-14 14:47:02,845 cisTarget INFO Getting cistromes for Topic7 2023-11-14 14:47:03,153 cisTarget INFO Running cisTarget for Topic8 which has 4326 regions 2023-11-14 14:47:23,048 cisTarget INFO Annotating motifs for Topic8 2023-11-14 14:47:26,690 cisTarget INFO Getting cistromes for Topic8 2023-11-14 14:47:27,137 cisTarget INFO Running cisTarget for Topic9 which has 4448 regions 2023-11-14 14:47:47,506 cisTarget INFO Annotating motifs for Topic9 2023-11-14 14:47:50,770 cisTarget INFO Getting cistromes for Topic9 2023-11-14 14:47:51,120 cisTarget INFO Running cisTarget for Topic10 which has 5533 regions 2023-11-14 14:48:11,100 cisTarget INFO Annotating motifs for Topic10 2023-11-14 14:48:14,493 cisTarget INFO Getting cistromes for Topic10 2023-11-14 14:48:14,909 cisTarget INFO Running cisTarget for Topic11 which has 3226 regions 2023-11-14 14:48:34,514 cisTarget INFO Annotating motifs for Topic11 2023-11-14 14:48:37,609 cisTarget INFO Getting cistromes for Topic11 2023-11-14 14:48:37,925 cisTarget INFO Running cisTarget for Topic12 which has 5220 regions 2023-11-14 14:48:59,401 cisTarget INFO Annotating motifs for Topic12 2023-11-14 14:49:03,088 cisTarget INFO Getting cistromes for Topic12 2023-11-14 14:49:03,481 cisTarget INFO Running cisTarget for Topic13 which has 855 regions 2023-11-14 14:49:22,157 cisTarget INFO Annotating motifs for Topic13 2023-11-14 14:49:24,842 cisTarget INFO Getting cistromes for Topic13 2023-11-14 14:49:25,233 cisTarget INFO Running cisTarget for Topic14 which has 704 regions 2023-11-14 14:49:44,226 cisTarget INFO Annotating motifs for Topic14 2023-11-14 14:49:47,018 cisTarget INFO Getting cistromes for Topic14 2023-11-14 14:49:47,526 cisTarget INFO Running cisTarget for Topic15 which has 3281 regions 2023-11-14 14:50:06,808 cisTarget INFO Annotating motifs for Topic15 2023-11-14 14:50:10,182 cisTarget INFO Getting cistromes for Topic15 2023-11-14 14:50:10,678 cisTarget INFO Running cisTarget for Topic16 which has 7734 regions 2023-11-14 14:50:32,461 cisTarget INFO Annotating motifs for Topic16 2023-11-14 14:50:38,245 cisTarget INFO Getting cistromes for Topic16 2023-11-14 14:50:38,818 cisTarget INFO Done! 2023-11-14 14:50:38,821 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_otsu_No_promoters folder already exists. 2023-11-14 14:50:39,748 pycisTarget_wrapper INFO Running DEM for topics_otsu 2023-11-14 14:50:39,749 DEM INFO Reading DEM database 2023-11-14 14:53:15,796 DEM INFO Creating contrast groups 2023-11-14 14:53:32,679 DEM INFO Computing DEM for Topic1 2023-11-14 14:53:44,185 DEM INFO Computing DEM for Topic2 2023-11-14 14:53:54,264 DEM INFO Computing DEM for Topic3 2023-11-14 14:54:05,413 DEM INFO Computing DEM for Topic4 2023-11-14 14:54:12,648 DEM INFO Computing DEM for Topic5 2023-11-14 14:54:55,365 DEM INFO Computing DEM for Topic6 2023-11-14 14:55:11,273 DEM INFO Computing DEM for Topic7 2023-11-14 14:55:19,533 DEM INFO Computing DEM for Topic8 2023-11-14 14:55:35,715 DEM INFO Computing DEM for Topic9 2023-11-14 14:55:55,462 DEM INFO Computing DEM for Topic10 2023-11-14 14:56:18,485 DEM INFO Computing DEM for Topic11 2023-11-14 14:56:56,854 DEM INFO Computing DEM for Topic12 2023-11-14 14:57:30,546 DEM INFO Computing DEM for Topic13 2023-11-14 14:57:52,154 DEM INFO Computing DEM for Topic14 2023-11-14 14:59:41,912 DEM INFO Computing DEM for Topic15 2023-11-14 15:00:04,310 DEM INFO Computing DEM for Topic16 2023-11-14 15:00:47,283 DEM INFO Forming cistromes 2023-11-14 15:00:55,930 DEM INFO Done! 2023-11-14 15:00:55,937 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_All folder already exists. 2023-11-14 15:00:56,981 pycisTarget_wrapper INFO Running DEM without promoters for topics_otsu 2023-11-14 15:01:10,046 DEM INFO Creating contrast groups 2023-11-14 15:01:12,071 DEM INFO Computing DEM for Topic1 2023-11-14 15:01:26,912 DEM INFO Computing DEM for Topic2 2023-11-14 15:01:35,275 DEM INFO Computing DEM for Topic3 2023-11-14 15:01:45,815 DEM INFO Computing DEM for Topic4 2023-11-14 15:01:53,043 DEM INFO Computing DEM for Topic5 2023-11-14 15:02:07,883 DEM INFO Computing DEM for Topic6 2023-11-14 15:02:18,473 DEM INFO Computing DEM for Topic7 2023-11-14 15:02:26,438 DEM INFO Computing DEM for Topic8 2023-11-14 15:02:40,317 DEM INFO Computing DEM for Topic9 2023-11-14 15:02:58,754 DEM INFO Computing DEM for Topic10 2023-11-14 15:03:11,774 DEM INFO Computing DEM for Topic11 2023-11-14 15:03:30,788 DEM INFO Computing DEM for Topic12 2023-11-14 15:03:43,488 DEM INFO Computing DEM for Topic13 2023-11-14 15:03:57,631 DEM INFO Computing DEM for Topic14 2023-11-14 15:04:21,460 DEM INFO Computing DEM for Topic15 2023-11-14 15:04:44,351 DEM INFO Computing DEM for Topic16 2023-11-14 15:05:15,287 DEM INFO Forming cistromes 2023-11-14 15:05:23,512 DEM INFO Done! 2023-11-14 15:05:23,517 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_No_promoters folder already exists. 2023-11-14 15:05:24,533 pycisTarget_wrapper INFO Loading cisTarget database for topics_top_3 2023-11-14 15:05:24,534 cisTarget INFO Reading cisTarget database 2023-11-14 15:08:37,317 pycisTarget_wrapper INFO Running cisTarget for topics_top_3 2023-11-14 15:08:37,320 cisTarget INFO Running cisTarget for Topic1 which has 3413 regions 2023-11-14 15:08:57,989 cisTarget INFO Annotating motifs for Topic1 2023-11-14 15:09:01,201 cisTarget INFO Getting cistromes for Topic1 2023-11-14 15:09:01,554 cisTarget INFO Running cisTarget for Topic2 which has 3125 regions 2023-11-14 15:09:19,476 cisTarget INFO Annotating motifs for Topic2 2023-11-14 15:09:22,177 cisTarget INFO Getting cistromes for Topic2 2023-11-14 15:09:22,653 cisTarget INFO Running cisTarget for Topic3 which has 3206 regions 2023-11-14 15:09:41,213 cisTarget INFO Annotating motifs for Topic3 2023-11-14 15:09:44,310 cisTarget INFO Getting cistromes for Topic3 2023-11-14 15:09:44,849 cisTarget INFO Running cisTarget for Topic4 which has 3529 regions 2023-11-14 15:10:03,381 cisTarget INFO Annotating motifs for Topic4 2023-11-14 15:10:06,329 cisTarget INFO Getting cistromes for Topic4 2023-11-14 15:10:06,806 cisTarget INFO Running cisTarget for Topic5 which has 3971 regions 2023-11-14 15:10:27,135 cisTarget INFO Annotating motifs for Topic5 2023-11-14 15:10:31,127 cisTarget INFO Getting cistromes for Topic5 2023-11-14 15:10:31,747 cisTarget INFO Running cisTarget for Topic6 which has 4058 regions 2023-11-14 15:10:50,719 cisTarget INFO Annotating motifs for Topic6 2023-11-14 15:10:54,190 cisTarget INFO Getting cistromes for Topic6 2023-11-14 15:10:54,873 cisTarget INFO Running cisTarget for Topic7 which has 3471 regions 2023-11-14 15:11:14,121 cisTarget INFO Annotating motifs for Topic7 2023-11-14 15:11:17,058 cisTarget INFO Getting cistromes for Topic7 2023-11-14 15:11:17,540 cisTarget INFO Running cisTarget for Topic8 which has 3560 regions 2023-11-14 15:11:38,052 cisTarget INFO Annotating motifs for Topic8 2023-11-14 15:11:41,340 cisTarget INFO Getting cistromes for Topic8 2023-11-14 15:11:41,848 cisTarget INFO Running cisTarget for Topic9 which has 3347 regions 2023-11-14 15:12:00,712 cisTarget INFO Annotating motifs for Topic9 2023-11-14 15:12:03,890 cisTarget INFO Getting cistromes for Topic9 2023-11-14 15:12:04,506 cisTarget INFO Running cisTarget for Topic10 which has 3397 regions 2023-11-14 15:12:22,434 cisTarget INFO Annotating motifs for Topic10 2023-11-14 15:12:25,345 cisTarget INFO Getting cistromes for Topic10 2023-11-14 15:12:25,715 cisTarget INFO Running cisTarget for Topic11 which has 3435 regions 2023-11-14 15:12:44,196 cisTarget INFO Annotating motifs for Topic11 2023-11-14 15:12:47,302 cisTarget INFO Getting cistromes for Topic11 2023-11-14 15:12:47,778 cisTarget INFO Running cisTarget for Topic12 which has 3393 regions 2023-11-14 15:13:06,405 cisTarget INFO Annotating motifs for Topic12 2023-11-14 15:13:09,663 cisTarget INFO Getting cistromes for Topic12 2023-11-14 15:13:10,300 cisTarget INFO Running cisTarget for Topic13 which has 4250 regions 2023-11-14 15:13:29,467 cisTarget INFO Annotating motifs for Topic13 2023-11-14 15:13:33,116 cisTarget INFO Getting cistromes for Topic13 2023-11-14 15:13:33,677 cisTarget INFO Running cisTarget for Topic14 which has 3853 regions 2023-11-14 15:13:53,639 cisTarget INFO Annotating motifs for Topic14 2023-11-14 15:13:57,655 cisTarget INFO Getting cistromes for Topic14 2023-11-14 15:13:58,322 cisTarget INFO Running cisTarget for Topic15 which has 3489 regions 2023-11-14 15:14:17,102 cisTarget INFO Annotating motifs for Topic15 2023-11-14 15:14:20,335 cisTarget INFO Getting cistromes for Topic15 2023-11-14 15:14:20,922 cisTarget INFO Running cisTarget for Topic16 which has 3428 regions 2023-11-14 15:14:39,389 cisTarget INFO Annotating motifs for Topic16 2023-11-14 15:14:42,491 cisTarget INFO Getting cistromes for Topic16 2023-11-14 15:14:43,169 cisTarget INFO Done! 2023-11-14 15:14:43,171 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_All folder already exists. 2023-11-14 15:14:44,234 pycisTarget_wrapper INFO Running cisTarget without promoters for topics_top_3 2023-11-14 15:14:53,974 cisTarget INFO Running cisTarget for Topic1 which has 2657 regions 2023-11-14 15:15:12,661 cisTarget INFO Annotating motifs for Topic1 2023-11-14 15:15:15,622 cisTarget INFO Getting cistromes for Topic1 2023-11-14 15:15:16,043 cisTarget INFO Running cisTarget for Topic2 which has 2369 regions 2023-11-14 15:15:33,105 cisTarget INFO Annotating motifs for Topic2 2023-11-14 15:15:35,660 cisTarget INFO Getting cistromes for Topic2 2023-11-14 15:15:35,985 cisTarget INFO Running cisTarget for Topic3 which has 2678 regions 2023-11-14 15:15:54,118 cisTarget INFO Annotating motifs for Topic3 2023-11-14 15:15:57,088 cisTarget INFO Getting cistromes for Topic3 2023-11-14 15:15:57,430 cisTarget INFO Running cisTarget for Topic4 which has 2527 regions 2023-11-14 15:16:15,376 cisTarget INFO Annotating motifs for Topic4 2023-11-14 15:16:18,151 cisTarget INFO Getting cistromes for Topic4 2023-11-14 15:16:18,501 cisTarget INFO Running cisTarget for Topic5 which has 567 regions 2023-11-14 15:16:36,033 cisTarget INFO Annotating motifs for Topic5 2023-11-14 15:16:38,623 cisTarget INFO Getting cistromes for Topic5 2023-11-14 15:16:39,141 cisTarget INFO Running cisTarget for Topic6 which has 1527 regions 2023-11-14 15:16:58,247 cisTarget INFO Annotating motifs for Topic6 2023-11-14 15:17:00,917 cisTarget INFO Getting cistromes for Topic6 2023-11-14 15:17:01,352 cisTarget INFO Running cisTarget for Topic7 which has 2394 regions 2023-11-14 15:17:18,215 cisTarget INFO Annotating motifs for Topic7 2023-11-14 15:17:20,749 cisTarget INFO Getting cistromes for Topic7 2023-11-14 15:17:21,015 cisTarget INFO Running cisTarget for Topic8 which has 2546 regions 2023-11-14 15:17:39,279 cisTarget INFO Annotating motifs for Topic8 2023-11-14 15:17:42,338 cisTarget INFO Getting cistromes for Topic8 2023-11-14 15:17:42,763 cisTarget INFO Running cisTarget for Topic9 which has 2262 regions 2023-11-14 15:18:02,598 cisTarget INFO Annotating motifs for Topic9 2023-11-14 15:18:05,499 cisTarget INFO Getting cistromes for Topic9 2023-11-14 15:18:05,807 cisTarget INFO Running cisTarget for Topic10 which has 2599 regions 2023-11-14 15:18:23,450 cisTarget INFO Annotating motifs for Topic10 2023-11-14 15:18:26,290 cisTarget INFO Getting cistromes for Topic10 2023-11-14 15:18:26,650 cisTarget INFO Running cisTarget for Topic11 which has 2562 regions 2023-11-14 15:18:45,195 cisTarget INFO Annotating motifs for Topic11 2023-11-14 15:18:48,162 cisTarget INFO Getting cistromes for Topic11 2023-11-14 15:18:48,491 cisTarget INFO Running cisTarget for Topic12 which has 2143 regions 2023-11-14 15:19:06,810 cisTarget INFO Annotating motifs for Topic12 2023-11-14 15:19:09,749 cisTarget INFO Getting cistromes for Topic12 2023-11-14 15:19:10,163 cisTarget INFO Running cisTarget for Topic13 which has 1375 regions 2023-11-14 15:19:27,967 cisTarget INFO Annotating motifs for Topic13 2023-11-14 15:19:30,642 cisTarget INFO Getting cistromes for Topic13 2023-11-14 15:19:31,109 cisTarget INFO Running cisTarget for Topic14 which has 329 regions 2023-11-14 15:19:49,560 cisTarget INFO Annotating motifs for Topic14 2023-11-14 15:19:52,297 cisTarget INFO Getting cistromes for Topic14 2023-11-14 15:19:52,794 cisTarget INFO Running cisTarget for Topic15 which has 2382 regions 2023-11-14 15:20:11,740 cisTarget INFO Annotating motifs for Topic15 2023-11-14 15:20:14,842 cisTarget INFO Getting cistromes for Topic15 2023-11-14 15:20:15,319 cisTarget INFO Running cisTarget for Topic16 which has 2785 regions 2023-11-14 15:20:34,905 cisTarget INFO Annotating motifs for Topic16 2023-11-14 15:20:37,960 cisTarget INFO Getting cistromes for Topic16 2023-11-14 15:20:38,368 cisTarget INFO Done! 2023-11-14 15:20:38,371 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_No_promoters folder already exists. 2023-11-14 15:20:39,161 pycisTarget_wrapper INFO Running DEM for topics_top_3 2023-11-14 15:20:39,162 DEM INFO Reading DEM database 2023-11-14 15:23:03,395 DEM INFO Creating contrast groups 2023-11-14 15:23:18,277 DEM INFO Computing DEM for Topic1 2023-11-14 15:23:30,539 DEM INFO Computing DEM for Topic2 2023-11-14 15:23:37,536 DEM INFO Computing DEM for Topic3 2023-11-14 15:23:48,353 DEM INFO Computing DEM for Topic4 2023-11-14 15:23:55,796 DEM INFO Computing DEM for Topic5 2023-11-14 15:24:45,513 DEM INFO Computing DEM for Topic6 2023-11-14 15:24:57,273 DEM INFO Computing DEM for Topic7 2023-11-14 15:25:04,635 DEM INFO Computing DEM for Topic8 2023-11-14 15:25:17,356 DEM INFO Computing DEM for Topic9 2023-11-14 15:25:34,283 DEM INFO Computing DEM for Topic10 2023-11-14 15:26:00,574 DEM INFO Computing DEM for Topic11 2023-11-14 15:26:15,807 DEM INFO Computing DEM for Topic12 2023-11-14 15:26:41,715 DEM INFO Computing DEM for Topic13 2023-11-14 15:27:00,831 DEM INFO Computing DEM for Topic14 2023-11-14 15:27:55,464 DEM INFO Computing DEM for Topic15 2023-11-14 15:28:05,908 DEM INFO Computing DEM for Topic16 2023-11-14 15:28:38,845 DEM INFO Forming cistromes 2023-11-14 15:28:45,751 DEM INFO Done! 2023-11-14 15:28:45,755 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_All folder already exists. 2023-11-14 15:28:46,741 pycisTarget_wrapper INFO Running DEM without promoters for topics_top_3 2023-11-14 15:28:56,871 DEM INFO Creating contrast groups 2023-11-14 15:28:58,153 DEM INFO Computing DEM for Topic1 2023-11-14 15:29:08,454 DEM INFO Computing DEM for Topic2 2023-11-14 15:29:15,488 DEM INFO Computing DEM for Topic3 2023-11-14 15:29:23,037 DEM INFO Computing DEM for Topic4 2023-11-14 15:29:29,224 DEM INFO Computing DEM for Topic5 2023-11-14 15:29:43,191 DEM INFO Computing DEM for Topic6 2023-11-14 15:29:54,836 DEM INFO Computing DEM for Topic7 2023-11-14 15:30:01,625 DEM INFO Computing DEM for Topic8 2023-11-14 15:30:08,730 DEM INFO Computing DEM for Topic9 2023-11-14 15:30:23,133 DEM INFO Computing DEM for Topic10 2023-11-14 15:30:39,920 DEM INFO Computing DEM for Topic11 2023-11-14 15:30:59,357 DEM INFO Computing DEM for Topic12 2023-11-14 15:31:12,615 DEM INFO Computing DEM for Topic13 2023-11-14 15:31:25,791 DEM INFO Computing DEM for Topic14 2023-11-14 15:31:41,341 DEM INFO Computing DEM for Topic15 2023-11-14 15:31:59,696 DEM INFO Computing DEM for Topic16 2023-11-14 15:32:22,851 DEM INFO Forming cistromes 2023-11-14 15:32:29,807 DEM INFO Done! 2023-11-14 15:32:29,811 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_No_promoters folder already exists. 2023-11-14 15:32:30,889 pycisTarget_wrapper INFO Loading cisTarget database for DARs 2023-11-14 15:32:30,890 cisTarget INFO Reading cisTarget database 2023-11-14 15:35:56,216 pycisTarget_wrapper INFO Running cisTarget for DARs 2023-11-14 15:35:56,220 cisTarget INFO Running cisTarget for B_cells which has 11050 regions 2023-11-14 15:36:20,092 cisTarget INFO Annotating motifs for B_cells 2023-11-14 15:36:24,410 cisTarget INFO Getting cistromes for B_cells 2023-11-14 15:36:24,903 cisTarget INFO Running cisTarget for B_cells_1 which has 30035 regions 2023-11-14 15:36:59,595 cisTarget INFO Annotating motifs for B_cells_1 2023-11-14 15:37:07,747 cisTarget INFO Getting cistromes for B_cells_1 2023-11-14 15:37:08,728 cisTarget INFO Running cisTarget for B_cells_2 which has 11872 regions 2023-11-14 15:37:31,067 cisTarget INFO Annotating motifs for B_cells_2 2023-11-14 15:37:35,096 cisTarget INFO Getting cistromes for B_cells_2 2023-11-14 15:37:35,564 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 29730 regions 2023-11-14 15:38:07,690 cisTarget INFO Annotating motifs for CD14+_Monocytes 2023-11-14 15:38:16,045 cisTarget INFO Getting cistromes for CD14+_Monocytes 2023-11-14 15:38:16,987 cisTarget INFO Running cisTarget for CD4_T_cells which has 16991 regions 2023-11-14 15:38:40,429 cisTarget INFO Annotating motifs for CD4_T_cells 2023-11-14 15:38:45,743 cisTarget INFO Getting cistromes for CD4_T_cells 2023-11-14 15:38:46,457 cisTarget INFO Running cisTarget for CD8_T_cells which has 9333 regions 2023-11-14 15:39:07,418 cisTarget INFO Annotating motifs for CD8_T_cells 2023-11-14 15:39:11,329 cisTarget INFO Getting cistromes for CD8_T_cells 2023-11-14 15:39:11,800 cisTarget INFO Running cisTarget for Dendritic_cells which has 24933 regions 2023-11-14 15:39:42,203 cisTarget INFO Annotating motifs for Dendritic_cells 2023-11-14 15:39:49,757 cisTarget INFO Getting cistromes for Dendritic_cells 2023-11-14 15:39:50,532 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 28776 regions 2023-11-14 15:40:21,710 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes 2023-11-14 15:40:29,709 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes 2023-11-14 15:40:30,598 cisTarget INFO Running cisTarget for NK_cells which has 7830 regions 2023-11-14 15:40:53,193 cisTarget INFO Annotating motifs for NK_cells 2023-11-14 15:40:56,845 cisTarget INFO Getting cistromes for NK_cells 2023-11-14 15:40:57,257 cisTarget INFO Done! 2023-11-14 15:40:57,259 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_All folder already exists. 2023-11-14 15:40:57,765 pycisTarget_wrapper INFO Running cisTarget without promoters for DARs 2023-11-14 15:41:08,368 cisTarget INFO Running cisTarget for B_cells which has 9314 regions 2023-11-14 15:41:29,938 cisTarget INFO Annotating motifs for B_cells 2023-11-14 15:41:33,897 cisTarget INFO Getting cistromes for B_cells 2023-11-14 15:41:34,334 cisTarget INFO Running cisTarget for B_cells_1 which has 27323 regions 2023-11-14 15:42:05,208 cisTarget INFO Annotating motifs for B_cells_1 2023-11-14 15:42:13,667 cisTarget INFO Getting cistromes for B_cells_1 2023-11-14 15:42:14,726 cisTarget INFO Running cisTarget for B_cells_2 which has 10305 regions 2023-11-14 15:42:39,156 cisTarget INFO Annotating motifs for B_cells_2 2023-11-14 15:42:43,544 cisTarget INFO Getting cistromes for B_cells_2 2023-11-14 15:42:43,976 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 26947 regions 2023-11-14 15:43:15,330 cisTarget INFO Annotating motifs for CD14+_Monocytes 2023-11-14 15:43:23,399 cisTarget INFO Getting cistromes for CD14+_Monocytes 2023-11-14 15:43:24,298 cisTarget INFO Running cisTarget for CD4_T_cells which has 12957 regions 2023-11-14 15:43:45,432 cisTarget INFO Annotating motifs for CD4_T_cells 2023-11-14 15:43:50,037 cisTarget INFO Getting cistromes for CD4_T_cells 2023-11-14 15:43:50,611 cisTarget INFO Running cisTarget for CD8_T_cells which has 7977 regions 2023-11-14 15:44:13,290 cisTarget INFO Annotating motifs for CD8_T_cells 2023-11-14 15:44:17,270 cisTarget INFO Getting cistromes for CD8_T_cells 2023-11-14 15:44:17,831 cisTarget INFO Running cisTarget for Dendritic_cells which has 22578 regions 2023-11-14 15:44:46,802 cisTarget INFO Annotating motifs for Dendritic_cells 2023-11-14 15:44:53,749 cisTarget INFO Getting cistromes for Dendritic_cells 2023-11-14 15:44:54,454 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 26272 regions 2023-11-14 15:45:24,249 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes 2023-11-14 15:45:31,902 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes 2023-11-14 15:45:32,802 cisTarget INFO Running cisTarget for NK_cells which has 6644 regions 2023-11-14 15:45:53,916 cisTarget INFO Annotating motifs for NK_cells 2023-11-14 15:45:57,420 cisTarget INFO Getting cistromes for NK_cells 2023-11-14 15:45:57,806 cisTarget INFO Done! 2023-11-14 15:45:57,809 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_No_promoters folder already exists. 2023-11-14 15:45:58,552 pycisTarget_wrapper INFO Running DEM for DARs 2023-11-14 15:45:58,553 DEM INFO Reading DEM database 2023-11-14 15:48:03,119 DEM INFO Creating contrast groups 2023-11-14 15:48:14,218 DEM INFO Computing DEM for B_cells 2023-11-14 15:48:30,338 DEM INFO Computing DEM for B_cells_1 2023-11-14 15:49:35,666 DEM INFO Computing DEM for B_cells_2 2023-11-14 15:49:56,010 DEM INFO Computing DEM for CD14+_Monocytes 2023-11-14 15:51:09,374 DEM INFO Computing DEM for CD4_T_cells 2023-11-14 15:52:25,022 DEM INFO Computing DEM for CD8_T_cells 2023-11-14 15:52:40,935 DEM INFO Computing DEM for Dendritic_cells 2023-11-14 15:53:25,143 DEM INFO Computing DEM for FCGR3A+_Monocytes 2023-11-14 15:54:46,429 DEM INFO Computing DEM for NK_cells 2023-11-14 15:55:00,798 DEM INFO Forming cistromes 2023-11-14 15:55:03,233 DEM INFO Done! 2023-11-14 15:55:03,240 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_All folder already exists. 2023-11-14 15:55:03,732 pycisTarget_wrapper INFO Running DEM without promoters for DARs 2023-11-14 15:55:14,195 DEM INFO Creating contrast groups 2023-11-14 15:55:16,705 DEM INFO Computing DEM for B_cells 2023-11-14 15:55:44,133 DEM INFO Computing DEM for B_cells_1 2023-11-14 15:56:42,689 DEM INFO Computing DEM for B_cells_2 2023-11-14 15:57:13,772 DEM INFO Computing DEM for CD14+_Monocytes 2023-11-14 15:58:08,363 DEM INFO Computing DEM for CD4_T_cells 2023-11-14 15:58:35,800 DEM INFO Computing DEM for CD8_T_cells 2023-11-14 15:58:53,365 DEM INFO Computing DEM for Dendritic_cells 2023-11-14 15:59:41,567 DEM INFO Computing DEM for FCGR3A+_Monocytes 2023-11-14 16:00:37,366 DEM INFO Computing DEM for NK_cells 2023-11-14 16:00:56,239 DEM INFO Forming cistromes 2023-11-14 16:00:58,656 DEM INFO Done! 2023-11-14 16:00:58,663 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_No_promoters folder already exists. 2023-11-14 16:00:59,267 pycisTarget_wrapper INFO Saving object

SeppeDeWinter commented 11 months ago

Hi @Chrisdoan9

I don't see any errors here. I guess you are still running out of memory ...

All the best,

Seppe

Chrisdoan9 commented 11 months ago

Hi @SeppeDeWinter,

Thank you for your suggestion. I put 1024Gb in memory, so may I know how much memory we need to run?

SeppeDeWinter commented 11 months ago

Hi @Chrisdoan9

1 Tb of memory should definitely be enough... you could try to monitor at which step it runs out of memory.

Also, you can try to use the development branch (see: https://github.com/aertslab/scenicplus/discussions/202) the code there is more memory efficient.

All the best,

Seppe

Chrisdoan9 commented 11 months ago

Hi @SeppeDeWinter

This is what I have in the end of the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

Would you please tell me how to run the development branch? Thank you. https://github.com/aertslab/scenicplus/tree/development

SeppeDeWinter commented 11 months ago

Hi @Chrisdoan9

Please refer to https://github.com/aertslab/scenicplus/discussions/202 for the use of the development branch. You can post question in that discussion thread and I will be happy to answer them.

All the best,

Seppe