search

aertslab / pySCENIC

pySCENIC is a lightning-fast python implementation of the SCENIC pipeline (Single-Cell rEgulatory Network Inference and Clustering) which enables biologists to infer transcription factors, gene regulatory networks and cell types from single-cell RNA-seq data.
http://scenic.aertslab.org
GNU General Public License v3.0
424 stars 179 forks source link

CTX running infinitely long. Defunct child processes #547

Closed arunchakra closed 4 months ago

arunchakra commented 4 months ago

Problem: When running pySCENIC on our HPC system using SLURM, a significant number of defunct (zombie) processes are being produced. These processes appear during the execution of the ctx module and persist throughout the runtime, not being cleaned up by the parent process. Thus the ctx run for infinitely long, because it seems the parent process is not handling the child processes. Looks like it doesn't get past the pyscenic.prune stage.

Tried using the ctx command line interface directly, and also tried using apptainer(singularity) container with the same issue. Used 16 CPUs with 64 GB of ram. Would run for >24 hours with only a few thousand cells.

Has anyone encountered this or is this a known issues with pySCENIC or specific configurations that might help mitigate this problem?

arunchakra commented 4 months ago

For some reason # of cores affect this behavior. Reduced number of cores to 8 or 6 and it worked fine