search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
165 stars 27 forks source link

Help with Kernal dead #259

Open Chrisdoan9 opened 7 months ago

Chrisdoan9 commented 7 months ago

Hi team,

Would you please suggest what cause the kernal dead error? Thank you so much! This cell ran for about 2 hours and then I got the error kernal dead. I put 1024Gb of memory and still got kernal dead.

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )

This is what I have in the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

I found a similar issue here: https://github.com/aertslab/SCENICprotocol/issues/28 I run Jupyter notebook on a server.

Version (please complete the following information):

SeppeDeWinter commented 7 months ago

Hi @Chrisdoan9

At what step is the kernel dying?

Also the development version of the code uses less memory, so you can also consider using that version.

See https://github.com/aertslab/scenicplus/discussions/202 for more info.

All the best,

Seppe

Chrisdoan9 commented 7 months ago

Hi @SeppeDeWinter

The kernel dying at this cell:

from scenicplus.wrappers.run_pycistarget import run_pycistarget
run_pycistarget(
    region_sets = region_sets,
    species = 'homo_sapiens',
    save_path = os.path.join(work_dir, 'motifs'),
    ctx_db_path = rankings_db,
    dem_db_path = scores_db,
    path_to_motif_annotations = motif_annotation,
    run_without_promoters = True,
    n_tpu = 1,
    _temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    annotation_version = 'v10nr_clust',
    )
SeppeDeWinter commented 7 months ago

Hi @Chrisdoan9

Did you have any output before the kernel died? And could you please copy-paste this? The function you pasted above will run many steps, so it's difficult for me to know what might be causing the issue.

Best,

Seppe

Chrisdoan9 commented 7 months ago

Hi @SeppeDeWinter, sure. That function printed a lot of output during about 2 hours of running:

2023-11-14 14:33:44,142 pycisTarget_wrapper INFO pbmc_tutorial/motifs folder already exists. 2023-11-14 14:33:45,422 pycisTarget_wrapper INFO Loading cisTarget database for topics_otsu 2023-11-14 14:33:45,423 cisTarget INFO Reading cisTarget database 2023-11-14 14:37:52,116 pycisTarget_wrapper INFO Running cisTarget for topics_otsu 2023-11-14 14:37:52,118 cisTarget INFO Running cisTarget for Topic1 which has 4811 regions 2023-11-14 14:38:15,144 cisTarget INFO Annotating motifs for Topic1 2023-11-14 14:38:18,547 cisTarget INFO Getting cistromes for Topic1 2023-11-14 14:38:18,939 cisTarget INFO Running cisTarget for Topic2 which has 7981 regions 2023-11-14 14:38:38,055 cisTarget INFO Annotating motifs for Topic2 2023-11-14 14:38:41,118 cisTarget INFO Getting cistromes for Topic2 2023-11-14 14:38:41,606 cisTarget INFO Running cisTarget for Topic3 which has 5233 regions 2023-11-14 14:39:01,134 cisTarget INFO Annotating motifs for Topic3 2023-11-14 14:39:04,537 cisTarget INFO Getting cistromes for Topic3 2023-11-14 14:39:05,020 cisTarget INFO Running cisTarget for Topic4 which has 3393 regions 2023-11-14 14:39:23,593 cisTarget INFO Annotating motifs for Topic4 2023-11-14 14:39:26,517 cisTarget INFO Getting cistromes for Topic4 2023-11-14 14:39:26,958 cisTarget INFO Running cisTarget for Topic5 which has 2608 regions 2023-11-14 14:39:45,569 cisTarget INFO Annotating motifs for Topic5 2023-11-14 14:39:49,003 cisTarget INFO Getting cistromes for Topic5 2023-11-14 14:39:49,659 cisTarget INFO Running cisTarget for Topic6 which has 1795 regions 2023-11-14 14:40:08,710 cisTarget INFO Annotating motifs for Topic6 2023-11-14 14:40:11,689 cisTarget INFO Getting cistromes for Topic6 2023-11-14 14:40:12,325 cisTarget INFO Running cisTarget for Topic7 which has 5383 regions 2023-11-14 14:40:31,390 cisTarget INFO Annotating motifs for Topic7 2023-11-14 14:40:34,348 cisTarget INFO Getting cistromes for Topic7 2023-11-14 14:40:34,863 cisTarget INFO Running cisTarget for Topic8 which has 6027 regions 2023-11-14 14:40:54,441 cisTarget INFO Annotating motifs for Topic8 2023-11-14 14:40:58,174 cisTarget INFO Getting cistromes for Topic8 2023-11-14 14:40:58,723 cisTarget INFO Running cisTarget for Topic9 which has 6579 regions 2023-11-14 14:41:18,911 cisTarget INFO Annotating motifs for Topic9 2023-11-14 14:41:22,620 cisTarget INFO Getting cistromes for Topic9 2023-11-14 14:41:23,350 cisTarget INFO Running cisTarget for Topic10 which has 7277 regions 2023-11-14 14:41:43,716 cisTarget INFO Annotating motifs for Topic10 2023-11-14 14:41:47,236 cisTarget INFO Getting cistromes for Topic10 2023-11-14 14:41:47,807 cisTarget INFO Running cisTarget for Topic11 which has 4394 regions 2023-11-14 14:42:06,965 cisTarget INFO Annotating motifs for Topic11 2023-11-14 14:42:10,277 cisTarget INFO Getting cistromes for Topic11 2023-11-14 14:42:10,882 cisTarget INFO Running cisTarget for Topic12 which has 8638 regions 2023-11-14 14:42:31,717 cisTarget INFO Annotating motifs for Topic12 2023-11-14 14:42:36,166 cisTarget INFO Getting cistromes for Topic12 2023-11-14 14:42:36,995 cisTarget INFO Running cisTarget for Topic13 which has 3038 regions 2023-11-14 14:42:56,722 cisTarget INFO Annotating motifs for Topic13 2023-11-14 14:43:00,047 cisTarget INFO Getting cistromes for Topic13 2023-11-14 14:43:00,567 cisTarget INFO Running cisTarget for Topic14 which has 6248 regions 2023-11-14 14:43:21,087 cisTarget INFO Annotating motifs for Topic14 2023-11-14 14:43:25,868 cisTarget INFO Getting cistromes for Topic14 2023-11-14 14:43:26,559 cisTarget INFO Running cisTarget for Topic15 which has 4824 regions 2023-11-14 14:43:46,414 cisTarget INFO Annotating motifs for Topic15 2023-11-14 14:43:49,907 cisTarget INFO Getting cistromes for Topic15 2023-11-14 14:43:50,422 cisTarget INFO Running cisTarget for Topic16 which has 9653 regions 2023-11-14 14:45:35,354 cisTarget INFO Getting cistromes for Topic3 2023-11-14 14:45:35,714 cisTarget INFO Running cisTarget for Topic4 which has 2426 regions 2023-11-14 14:45:53,764 cisTarget INFO Annotating motifs for Topic4 2023-11-14 14:45:56,499 cisTarget INFO Getting cistromes for Topic4 2023-11-14 14:45:56,838 cisTarget INFO Running cisTarget for Topic5 which has 277 regions 2023-11-14 14:46:15,899 cisTarget INFO Annotating motifs for Topic5 2023-11-14 14:46:18,438 cisTarget INFO Getting cistromes for Topic5 2023-11-14 14:46:18,892 cisTarget INFO Running cisTarget for Topic6 which has 512 regions 2023-11-14 14:46:38,179 cisTarget INFO Annotating motifs for Topic6 2023-11-14 14:46:40,905 cisTarget INFO Getting cistromes for Topic6 2023-11-14 14:46:41,348 cisTarget INFO Running cisTarget for Topic7 which has 3876 regions 2023-11-14 14:47:00,221 cisTarget INFO Annotating motifs for Topic7 2023-11-14 14:47:02,845 cisTarget INFO Getting cistromes for Topic7 2023-11-14 14:47:03,153 cisTarget INFO Running cisTarget for Topic8 which has 4326 regions 2023-11-14 14:47:23,048 cisTarget INFO Annotating motifs for Topic8 2023-11-14 14:47:26,690 cisTarget INFO Getting cistromes for Topic8 2023-11-14 14:47:27,137 cisTarget INFO Running cisTarget for Topic9 which has 4448 regions 2023-11-14 14:47:47,506 cisTarget INFO Annotating motifs for Topic9 2023-11-14 14:47:50,770 cisTarget INFO Getting cistromes for Topic9 2023-11-14 14:47:51,120 cisTarget INFO Running cisTarget for Topic10 which has 5533 regions 2023-11-14 14:48:11,100 cisTarget INFO Annotating motifs for Topic10 2023-11-14 14:48:14,493 cisTarget INFO Getting cistromes for Topic10 2023-11-14 14:48:14,909 cisTarget INFO Running cisTarget for Topic11 which has 3226 regions 2023-11-14 14:48:34,514 cisTarget INFO Annotating motifs for Topic11 2023-11-14 14:48:37,609 cisTarget INFO Getting cistromes for Topic11 2023-11-14 14:48:37,925 cisTarget INFO Running cisTarget for Topic12 which has 5220 regions 2023-11-14 14:48:59,401 cisTarget INFO Annotating motifs for Topic12 2023-11-14 14:49:03,088 cisTarget INFO Getting cistromes for Topic12 2023-11-14 14:49:03,481 cisTarget INFO Running cisTarget for Topic13 which has 855 regions 2023-11-14 14:49:22,157 cisTarget INFO Annotating motifs for Topic13 2023-11-14 14:49:24,842 cisTarget INFO Getting cistromes for Topic13 2023-11-14 14:49:25,233 cisTarget INFO Running cisTarget for Topic14 which has 704 regions 2023-11-14 14:49:44,226 cisTarget INFO Annotating motifs for Topic14 2023-11-14 14:49:47,018 cisTarget INFO Getting cistromes for Topic14 2023-11-14 14:49:47,526 cisTarget INFO Running cisTarget for Topic15 which has 3281 regions 2023-11-14 14:50:06,808 cisTarget INFO Annotating motifs for Topic15 2023-11-14 14:50:10,182 cisTarget INFO Getting cistromes for Topic15 2023-11-14 14:50:10,678 cisTarget INFO Running cisTarget for Topic16 which has 7734 regions 2023-11-14 14:50:32,461 cisTarget INFO Annotating motifs for Topic16 2023-11-14 14:50:38,245 cisTarget INFO Getting cistromes for Topic16 2023-11-14 14:50:38,818 cisTarget INFO Done! 2023-11-14 14:50:38,821 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_otsu_No_promoters folder already exists. 2023-11-14 14:50:39,748 pycisTarget_wrapper INFO Running DEM for topics_otsu 2023-11-14 14:50:39,749 DEM INFO Reading DEM database 2023-11-14 14:53:15,796 DEM INFO Creating contrast groups 2023-11-14 14:53:32,679 DEM INFO Computing DEM for Topic1 2023-11-14 14:53:44,185 DEM INFO Computing DEM for Topic2 2023-11-14 14:53:54,264 DEM INFO Computing DEM for Topic3 2023-11-14 14:54:05,413 DEM INFO Computing DEM for Topic4 2023-11-14 14:54:12,648 DEM INFO Computing DEM for Topic5 2023-11-14 14:54:55,365 DEM INFO Computing DEM for Topic6 2023-11-14 14:55:11,273 DEM INFO Computing DEM for Topic7 2023-11-14 14:55:19,533 DEM INFO Computing DEM for Topic8 2023-11-14 14:55:35,715 DEM INFO Computing DEM for Topic9 2023-11-14 14:55:55,462 DEM INFO Computing DEM for Topic10 2023-11-14 14:56:18,485 DEM INFO Computing DEM for Topic11 2023-11-14 14:56:56,854 DEM INFO Computing DEM for Topic12 2023-11-14 14:57:30,546 DEM INFO Computing DEM for Topic13 2023-11-14 14:57:52,154 DEM INFO Computing DEM for Topic14 2023-11-14 14:59:41,912 DEM INFO Computing DEM for Topic15 2023-11-14 15:00:04,310 DEM INFO Computing DEM for Topic16 2023-11-14 15:00:47,283 DEM INFO Forming cistromes 2023-11-14 15:00:55,930 DEM INFO Done! 2023-11-14 15:00:55,937 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_All folder already exists. 2023-11-14 15:00:56,981 pycisTarget_wrapper INFO Running DEM without promoters for topics_otsu 2023-11-14 15:01:10,046 DEM INFO Creating contrast groups 2023-11-14 15:01:12,071 DEM INFO Computing DEM for Topic1 2023-11-14 15:01:26,912 DEM INFO Computing DEM for Topic2 2023-11-14 15:01:35,275 DEM INFO Computing DEM for Topic3 2023-11-14 15:01:45,815 DEM INFO Computing DEM for Topic4 2023-11-14 15:01:53,043 DEM INFO Computing DEM for Topic5 2023-11-14 15:02:07,883 DEM INFO Computing DEM for Topic6 2023-11-14 15:02:18,473 DEM INFO Computing DEM for Topic7 2023-11-14 15:02:26,438 DEM INFO Computing DEM for Topic8 2023-11-14 15:02:40,317 DEM INFO Computing DEM for Topic9 2023-11-14 15:02:58,754 DEM INFO Computing DEM for Topic10 2023-11-14 15:03:11,774 DEM INFO Computing DEM for Topic11 2023-11-14 15:03:30,788 DEM INFO Computing DEM for Topic12 2023-11-14 15:03:43,488 DEM INFO Computing DEM for Topic13 2023-11-14 15:03:57,631 DEM INFO Computing DEM for Topic14 2023-11-14 15:04:21,460 DEM INFO Computing DEM for Topic15 2023-11-14 15:04:44,351 DEM INFO Computing DEM for Topic16 2023-11-14 15:05:15,287 DEM INFO Forming cistromes 2023-11-14 15:05:23,512 DEM INFO Done! 2023-11-14 15:05:23,517 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_otsu_No_promoters folder already exists. 2023-11-14 15:05:24,533 pycisTarget_wrapper INFO Loading cisTarget database for topics_top_3 2023-11-14 15:05:24,534 cisTarget INFO Reading cisTarget database 2023-11-14 15:08:37,317 pycisTarget_wrapper INFO Running cisTarget for topics_top_3 2023-11-14 15:08:37,320 cisTarget INFO Running cisTarget for Topic1 which has 3413 regions 2023-11-14 15:08:57,989 cisTarget INFO Annotating motifs for Topic1 2023-11-14 15:09:01,201 cisTarget INFO Getting cistromes for Topic1 2023-11-14 15:09:01,554 cisTarget INFO Running cisTarget for Topic2 which has 3125 regions 2023-11-14 15:09:19,476 cisTarget INFO Annotating motifs for Topic2 2023-11-14 15:09:22,177 cisTarget INFO Getting cistromes for Topic2 2023-11-14 15:09:22,653 cisTarget INFO Running cisTarget for Topic3 which has 3206 regions 2023-11-14 15:09:41,213 cisTarget INFO Annotating motifs for Topic3 2023-11-14 15:09:44,310 cisTarget INFO Getting cistromes for Topic3 2023-11-14 15:09:44,849 cisTarget INFO Running cisTarget for Topic4 which has 3529 regions 2023-11-14 15:10:03,381 cisTarget INFO Annotating motifs for Topic4 2023-11-14 15:10:06,329 cisTarget INFO Getting cistromes for Topic4 2023-11-14 15:10:06,806 cisTarget INFO Running cisTarget for Topic5 which has 3971 regions 2023-11-14 15:10:27,135 cisTarget INFO Annotating motifs for Topic5 2023-11-14 15:10:31,127 cisTarget INFO Getting cistromes for Topic5 2023-11-14 15:10:31,747 cisTarget INFO Running cisTarget for Topic6 which has 4058 regions 2023-11-14 15:10:50,719 cisTarget INFO Annotating motifs for Topic6 2023-11-14 15:10:54,190 cisTarget INFO Getting cistromes for Topic6 2023-11-14 15:10:54,873 cisTarget INFO Running cisTarget for Topic7 which has 3471 regions 2023-11-14 15:11:14,121 cisTarget INFO Annotating motifs for Topic7 2023-11-14 15:11:17,058 cisTarget INFO Getting cistromes for Topic7 2023-11-14 15:11:17,540 cisTarget INFO Running cisTarget for Topic8 which has 3560 regions 2023-11-14 15:11:38,052 cisTarget INFO Annotating motifs for Topic8 2023-11-14 15:11:41,340 cisTarget INFO Getting cistromes for Topic8 2023-11-14 15:11:41,848 cisTarget INFO Running cisTarget for Topic9 which has 3347 regions 2023-11-14 15:12:00,712 cisTarget INFO Annotating motifs for Topic9 2023-11-14 15:12:03,890 cisTarget INFO Getting cistromes for Topic9 2023-11-14 15:12:04,506 cisTarget INFO Running cisTarget for Topic10 which has 3397 regions 2023-11-14 15:12:22,434 cisTarget INFO Annotating motifs for Topic10 2023-11-14 15:12:25,345 cisTarget INFO Getting cistromes for Topic10 2023-11-14 15:12:25,715 cisTarget INFO Running cisTarget for Topic11 which has 3435 regions 2023-11-14 15:12:44,196 cisTarget INFO Annotating motifs for Topic11 2023-11-14 15:12:47,302 cisTarget INFO Getting cistromes for Topic11 2023-11-14 15:12:47,778 cisTarget INFO Running cisTarget for Topic12 which has 3393 regions 2023-11-14 15:13:06,405 cisTarget INFO Annotating motifs for Topic12 2023-11-14 15:13:09,663 cisTarget INFO Getting cistromes for Topic12 2023-11-14 15:13:10,300 cisTarget INFO Running cisTarget for Topic13 which has 4250 regions 2023-11-14 15:13:29,467 cisTarget INFO Annotating motifs for Topic13 2023-11-14 15:13:33,116 cisTarget INFO Getting cistromes for Topic13 2023-11-14 15:13:33,677 cisTarget INFO Running cisTarget for Topic14 which has 3853 regions 2023-11-14 15:13:53,639 cisTarget INFO Annotating motifs for Topic14 2023-11-14 15:13:57,655 cisTarget INFO Getting cistromes for Topic14 2023-11-14 15:13:58,322 cisTarget INFO Running cisTarget for Topic15 which has 3489 regions 2023-11-14 15:14:17,102 cisTarget INFO Annotating motifs for Topic15 2023-11-14 15:14:20,335 cisTarget INFO Getting cistromes for Topic15 2023-11-14 15:14:20,922 cisTarget INFO Running cisTarget for Topic16 which has 3428 regions 2023-11-14 15:14:39,389 cisTarget INFO Annotating motifs for Topic16 2023-11-14 15:14:42,491 cisTarget INFO Getting cistromes for Topic16 2023-11-14 15:14:43,169 cisTarget INFO Done! 2023-11-14 15:14:43,171 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_All folder already exists. 2023-11-14 15:14:44,234 pycisTarget_wrapper INFO Running cisTarget without promoters for topics_top_3 2023-11-14 15:14:53,974 cisTarget INFO Running cisTarget for Topic1 which has 2657 regions 2023-11-14 15:15:12,661 cisTarget INFO Annotating motifs for Topic1 2023-11-14 15:15:15,622 cisTarget INFO Getting cistromes for Topic1 2023-11-14 15:15:16,043 cisTarget INFO Running cisTarget for Topic2 which has 2369 regions 2023-11-14 15:15:33,105 cisTarget INFO Annotating motifs for Topic2 2023-11-14 15:15:35,660 cisTarget INFO Getting cistromes for Topic2 2023-11-14 15:15:35,985 cisTarget INFO Running cisTarget for Topic3 which has 2678 regions 2023-11-14 15:15:54,118 cisTarget INFO Annotating motifs for Topic3 2023-11-14 15:15:57,088 cisTarget INFO Getting cistromes for Topic3 2023-11-14 15:15:57,430 cisTarget INFO Running cisTarget for Topic4 which has 2527 regions 2023-11-14 15:16:15,376 cisTarget INFO Annotating motifs for Topic4 2023-11-14 15:16:18,151 cisTarget INFO Getting cistromes for Topic4 2023-11-14 15:16:18,501 cisTarget INFO Running cisTarget for Topic5 which has 567 regions 2023-11-14 15:16:36,033 cisTarget INFO Annotating motifs for Topic5 2023-11-14 15:16:38,623 cisTarget INFO Getting cistromes for Topic5 2023-11-14 15:16:39,141 cisTarget INFO Running cisTarget for Topic6 which has 1527 regions 2023-11-14 15:16:58,247 cisTarget INFO Annotating motifs for Topic6 2023-11-14 15:17:00,917 cisTarget INFO Getting cistromes for Topic6 2023-11-14 15:17:01,352 cisTarget INFO Running cisTarget for Topic7 which has 2394 regions 2023-11-14 15:17:18,215 cisTarget INFO Annotating motifs for Topic7 2023-11-14 15:17:20,749 cisTarget INFO Getting cistromes for Topic7 2023-11-14 15:17:21,015 cisTarget INFO Running cisTarget for Topic8 which has 2546 regions 2023-11-14 15:17:39,279 cisTarget INFO Annotating motifs for Topic8 2023-11-14 15:17:42,338 cisTarget INFO Getting cistromes for Topic8 2023-11-14 15:17:42,763 cisTarget INFO Running cisTarget for Topic9 which has 2262 regions 2023-11-14 15:18:02,598 cisTarget INFO Annotating motifs for Topic9 2023-11-14 15:18:05,499 cisTarget INFO Getting cistromes for Topic9 2023-11-14 15:18:05,807 cisTarget INFO Running cisTarget for Topic10 which has 2599 regions 2023-11-14 15:18:23,450 cisTarget INFO Annotating motifs for Topic10 2023-11-14 15:18:26,290 cisTarget INFO Getting cistromes for Topic10 2023-11-14 15:18:26,650 cisTarget INFO Running cisTarget for Topic11 which has 2562 regions 2023-11-14 15:18:45,195 cisTarget INFO Annotating motifs for Topic11 2023-11-14 15:18:48,162 cisTarget INFO Getting cistromes for Topic11 2023-11-14 15:18:48,491 cisTarget INFO Running cisTarget for Topic12 which has 2143 regions 2023-11-14 15:19:06,810 cisTarget INFO Annotating motifs for Topic12 2023-11-14 15:19:09,749 cisTarget INFO Getting cistromes for Topic12 2023-11-14 15:19:10,163 cisTarget INFO Running cisTarget for Topic13 which has 1375 regions 2023-11-14 15:19:27,967 cisTarget INFO Annotating motifs for Topic13 2023-11-14 15:19:30,642 cisTarget INFO Getting cistromes for Topic13 2023-11-14 15:19:31,109 cisTarget INFO Running cisTarget for Topic14 which has 329 regions 2023-11-14 15:19:49,560 cisTarget INFO Annotating motifs for Topic14 2023-11-14 15:19:52,297 cisTarget INFO Getting cistromes for Topic14 2023-11-14 15:19:52,794 cisTarget INFO Running cisTarget for Topic15 which has 2382 regions 2023-11-14 15:20:11,740 cisTarget INFO Annotating motifs for Topic15 2023-11-14 15:20:14,842 cisTarget INFO Getting cistromes for Topic15 2023-11-14 15:20:15,319 cisTarget INFO Running cisTarget for Topic16 which has 2785 regions 2023-11-14 15:20:34,905 cisTarget INFO Annotating motifs for Topic16 2023-11-14 15:20:37,960 cisTarget INFO Getting cistromes for Topic16 2023-11-14 15:20:38,368 cisTarget INFO Done! 2023-11-14 15:20:38,371 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_topics_top_3_No_promoters folder already exists. 2023-11-14 15:20:39,161 pycisTarget_wrapper INFO Running DEM for topics_top_3 2023-11-14 15:20:39,162 DEM INFO Reading DEM database 2023-11-14 15:23:03,395 DEM INFO Creating contrast groups 2023-11-14 15:23:18,277 DEM INFO Computing DEM for Topic1 2023-11-14 15:23:30,539 DEM INFO Computing DEM for Topic2 2023-11-14 15:23:37,536 DEM INFO Computing DEM for Topic3 2023-11-14 15:23:48,353 DEM INFO Computing DEM for Topic4 2023-11-14 15:23:55,796 DEM INFO Computing DEM for Topic5 2023-11-14 15:24:45,513 DEM INFO Computing DEM for Topic6 2023-11-14 15:24:57,273 DEM INFO Computing DEM for Topic7 2023-11-14 15:25:04,635 DEM INFO Computing DEM for Topic8 2023-11-14 15:25:17,356 DEM INFO Computing DEM for Topic9 2023-11-14 15:25:34,283 DEM INFO Computing DEM for Topic10 2023-11-14 15:26:00,574 DEM INFO Computing DEM for Topic11 2023-11-14 15:26:15,807 DEM INFO Computing DEM for Topic12 2023-11-14 15:26:41,715 DEM INFO Computing DEM for Topic13 2023-11-14 15:27:00,831 DEM INFO Computing DEM for Topic14 2023-11-14 15:27:55,464 DEM INFO Computing DEM for Topic15 2023-11-14 15:28:05,908 DEM INFO Computing DEM for Topic16 2023-11-14 15:28:38,845 DEM INFO Forming cistromes 2023-11-14 15:28:45,751 DEM INFO Done! 2023-11-14 15:28:45,755 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_All folder already exists. 2023-11-14 15:28:46,741 pycisTarget_wrapper INFO Running DEM without promoters for topics_top_3 2023-11-14 15:28:56,871 DEM INFO Creating contrast groups 2023-11-14 15:28:58,153 DEM INFO Computing DEM for Topic1 2023-11-14 15:29:08,454 DEM INFO Computing DEM for Topic2 2023-11-14 15:29:15,488 DEM INFO Computing DEM for Topic3 2023-11-14 15:29:23,037 DEM INFO Computing DEM for Topic4 2023-11-14 15:29:29,224 DEM INFO Computing DEM for Topic5 2023-11-14 15:29:43,191 DEM INFO Computing DEM for Topic6 2023-11-14 15:29:54,836 DEM INFO Computing DEM for Topic7 2023-11-14 15:30:01,625 DEM INFO Computing DEM for Topic8 2023-11-14 15:30:08,730 DEM INFO Computing DEM for Topic9 2023-11-14 15:30:23,133 DEM INFO Computing DEM for Topic10 2023-11-14 15:30:39,920 DEM INFO Computing DEM for Topic11 2023-11-14 15:30:59,357 DEM INFO Computing DEM for Topic12 2023-11-14 15:31:12,615 DEM INFO Computing DEM for Topic13 2023-11-14 15:31:25,791 DEM INFO Computing DEM for Topic14 2023-11-14 15:31:41,341 DEM INFO Computing DEM for Topic15 2023-11-14 15:31:59,696 DEM INFO Computing DEM for Topic16 2023-11-14 15:32:22,851 DEM INFO Forming cistromes 2023-11-14 15:32:29,807 DEM INFO Done! 2023-11-14 15:32:29,811 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_topics_top_3_No_promoters folder already exists. 2023-11-14 15:32:30,889 pycisTarget_wrapper INFO Loading cisTarget database for DARs 2023-11-14 15:32:30,890 cisTarget INFO Reading cisTarget database 2023-11-14 15:35:56,216 pycisTarget_wrapper INFO Running cisTarget for DARs 2023-11-14 15:35:56,220 cisTarget INFO Running cisTarget for B_cells which has 11050 regions 2023-11-14 15:36:20,092 cisTarget INFO Annotating motifs for B_cells 2023-11-14 15:36:24,410 cisTarget INFO Getting cistromes for B_cells 2023-11-14 15:36:24,903 cisTarget INFO Running cisTarget for B_cells_1 which has 30035 regions 2023-11-14 15:36:59,595 cisTarget INFO Annotating motifs for B_cells_1 2023-11-14 15:37:07,747 cisTarget INFO Getting cistromes for B_cells_1 2023-11-14 15:37:08,728 cisTarget INFO Running cisTarget for B_cells_2 which has 11872 regions 2023-11-14 15:37:31,067 cisTarget INFO Annotating motifs for B_cells_2 2023-11-14 15:37:35,096 cisTarget INFO Getting cistromes for B_cells_2 2023-11-14 15:37:35,564 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 29730 regions 2023-11-14 15:38:07,690 cisTarget INFO Annotating motifs for CD14+_Monocytes 2023-11-14 15:38:16,045 cisTarget INFO Getting cistromes for CD14+_Monocytes 2023-11-14 15:38:16,987 cisTarget INFO Running cisTarget for CD4_T_cells which has 16991 regions 2023-11-14 15:38:40,429 cisTarget INFO Annotating motifs for CD4_T_cells 2023-11-14 15:38:45,743 cisTarget INFO Getting cistromes for CD4_T_cells 2023-11-14 15:38:46,457 cisTarget INFO Running cisTarget for CD8_T_cells which has 9333 regions 2023-11-14 15:39:07,418 cisTarget INFO Annotating motifs for CD8_T_cells 2023-11-14 15:39:11,329 cisTarget INFO Getting cistromes for CD8_T_cells 2023-11-14 15:39:11,800 cisTarget INFO Running cisTarget for Dendritic_cells which has 24933 regions 2023-11-14 15:39:42,203 cisTarget INFO Annotating motifs for Dendritic_cells 2023-11-14 15:39:49,757 cisTarget INFO Getting cistromes for Dendritic_cells 2023-11-14 15:39:50,532 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 28776 regions 2023-11-14 15:40:21,710 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes 2023-11-14 15:40:29,709 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes 2023-11-14 15:40:30,598 cisTarget INFO Running cisTarget for NK_cells which has 7830 regions 2023-11-14 15:40:53,193 cisTarget INFO Annotating motifs for NK_cells 2023-11-14 15:40:56,845 cisTarget INFO Getting cistromes for NK_cells 2023-11-14 15:40:57,257 cisTarget INFO Done! 2023-11-14 15:40:57,259 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_All folder already exists. 2023-11-14 15:40:57,765 pycisTarget_wrapper INFO Running cisTarget without promoters for DARs 2023-11-14 15:41:08,368 cisTarget INFO Running cisTarget for B_cells which has 9314 regions 2023-11-14 15:41:29,938 cisTarget INFO Annotating motifs for B_cells 2023-11-14 15:41:33,897 cisTarget INFO Getting cistromes for B_cells 2023-11-14 15:41:34,334 cisTarget INFO Running cisTarget for B_cells_1 which has 27323 regions 2023-11-14 15:42:05,208 cisTarget INFO Annotating motifs for B_cells_1 2023-11-14 15:42:13,667 cisTarget INFO Getting cistromes for B_cells_1 2023-11-14 15:42:14,726 cisTarget INFO Running cisTarget for B_cells_2 which has 10305 regions 2023-11-14 15:42:39,156 cisTarget INFO Annotating motifs for B_cells_2 2023-11-14 15:42:43,544 cisTarget INFO Getting cistromes for B_cells_2 2023-11-14 15:42:43,976 cisTarget INFO Running cisTarget for CD14+_Monocytes which has 26947 regions 2023-11-14 15:43:15,330 cisTarget INFO Annotating motifs for CD14+_Monocytes 2023-11-14 15:43:23,399 cisTarget INFO Getting cistromes for CD14+_Monocytes 2023-11-14 15:43:24,298 cisTarget INFO Running cisTarget for CD4_T_cells which has 12957 regions 2023-11-14 15:43:45,432 cisTarget INFO Annotating motifs for CD4_T_cells 2023-11-14 15:43:50,037 cisTarget INFO Getting cistromes for CD4_T_cells 2023-11-14 15:43:50,611 cisTarget INFO Running cisTarget for CD8_T_cells which has 7977 regions 2023-11-14 15:44:13,290 cisTarget INFO Annotating motifs for CD8_T_cells 2023-11-14 15:44:17,270 cisTarget INFO Getting cistromes for CD8_T_cells 2023-11-14 15:44:17,831 cisTarget INFO Running cisTarget for Dendritic_cells which has 22578 regions 2023-11-14 15:44:46,802 cisTarget INFO Annotating motifs for Dendritic_cells 2023-11-14 15:44:53,749 cisTarget INFO Getting cistromes for Dendritic_cells 2023-11-14 15:44:54,454 cisTarget INFO Running cisTarget for FCGR3A+_Monocytes which has 26272 regions 2023-11-14 15:45:24,249 cisTarget INFO Annotating motifs for FCGR3A+_Monocytes 2023-11-14 15:45:31,902 cisTarget INFO Getting cistromes for FCGR3A+_Monocytes 2023-11-14 15:45:32,802 cisTarget INFO Running cisTarget for NK_cells which has 6644 regions 2023-11-14 15:45:53,916 cisTarget INFO Annotating motifs for NK_cells 2023-11-14 15:45:57,420 cisTarget INFO Getting cistromes for NK_cells 2023-11-14 15:45:57,806 cisTarget INFO Done! 2023-11-14 15:45:57,809 pycisTarget_wrapper INFO pbmc_tutorial/motifs/CTX_DARs_No_promoters folder already exists. 2023-11-14 15:45:58,552 pycisTarget_wrapper INFO Running DEM for DARs 2023-11-14 15:45:58,553 DEM INFO Reading DEM database 2023-11-14 15:48:03,119 DEM INFO Creating contrast groups 2023-11-14 15:48:14,218 DEM INFO Computing DEM for B_cells 2023-11-14 15:48:30,338 DEM INFO Computing DEM for B_cells_1 2023-11-14 15:49:35,666 DEM INFO Computing DEM for B_cells_2 2023-11-14 15:49:56,010 DEM INFO Computing DEM for CD14+_Monocytes 2023-11-14 15:51:09,374 DEM INFO Computing DEM for CD4_T_cells 2023-11-14 15:52:25,022 DEM INFO Computing DEM for CD8_T_cells 2023-11-14 15:52:40,935 DEM INFO Computing DEM for Dendritic_cells 2023-11-14 15:53:25,143 DEM INFO Computing DEM for FCGR3A+_Monocytes 2023-11-14 15:54:46,429 DEM INFO Computing DEM for NK_cells 2023-11-14 15:55:00,798 DEM INFO Forming cistromes 2023-11-14 15:55:03,233 DEM INFO Done! 2023-11-14 15:55:03,240 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_All folder already exists. 2023-11-14 15:55:03,732 pycisTarget_wrapper INFO Running DEM without promoters for DARs 2023-11-14 15:55:14,195 DEM INFO Creating contrast groups 2023-11-14 15:55:16,705 DEM INFO Computing DEM for B_cells 2023-11-14 15:55:44,133 DEM INFO Computing DEM for B_cells_1 2023-11-14 15:56:42,689 DEM INFO Computing DEM for B_cells_2 2023-11-14 15:57:13,772 DEM INFO Computing DEM for CD14+_Monocytes 2023-11-14 15:58:08,363 DEM INFO Computing DEM for CD4_T_cells 2023-11-14 15:58:35,800 DEM INFO Computing DEM for CD8_T_cells 2023-11-14 15:58:53,365 DEM INFO Computing DEM for Dendritic_cells 2023-11-14 15:59:41,567 DEM INFO Computing DEM for FCGR3A+_Monocytes 2023-11-14 16:00:37,366 DEM INFO Computing DEM for NK_cells 2023-11-14 16:00:56,239 DEM INFO Forming cistromes 2023-11-14 16:00:58,656 DEM INFO Done! 2023-11-14 16:00:58,663 pycisTarget_wrapper INFO pbmc_tutorial/motifs/DEM_DARs_No_promoters folder already exists. 2023-11-14 16:00:59,267 pycisTarget_wrapper INFO Saving object

SeppeDeWinter commented 7 months ago

Hi @Chrisdoan9

I don't see any errors here. I guess you are still running out of memory ...

All the best,

Seppe

Chrisdoan9 commented 7 months ago

Hi @SeppeDeWinter,

Thank you for your suggestion. I put 1024Gb in memory, so may I know how much memory we need to run?

SeppeDeWinter commented 7 months ago

Hi @Chrisdoan9

1 Tb of memory should definitely be enough... you could try to monitor at which step it runs out of memory.

Also, you can try to use the development branch (see: https://github.com/aertslab/scenicplus/discussions/202) the code there is more memory efficient.

All the best,

Seppe

Chrisdoan9 commented 6 months ago

Hi @SeppeDeWinter

This is what I have in the end of the log file:

[I 16:02:55.296 NotebookApp] Saving file at /project/multiome.ipynb
[I 16:16:57.060 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
kernel 0fc1137e-0f27-41a3-a544-4bd674cce75b restarted
[I 16:18:55.288 NotebookApp] Saving file at /project/multiome.ipynb

Would you please tell me how to run the development branch? Thank you. https://github.com/aertslab/scenicplus/tree/development

SeppeDeWinter commented 6 months ago

Hi @Chrisdoan9

Please refer to https://github.com/aertslab/scenicplus/discussions/202 for the use of the development branch. You can post question in that discussion thread and I will be happy to answer them.

All the best,

Seppe