PacificBiosciences / pb-CpG-tools

Collection of tools for the analysis of CpG data
BSD 3-Clause Clear License
70 stars 6 forks source link

Error: concurrent.futures.process.BrokenProcessPool #31

Closed yimingzhu100 closed 2 years ago

yimingzhu100 commented 2 years ago

Hi,

See below for the command I ran and the error that keeps showing up when it gets to ~80% completion. I wonder if this is a bug or user error. I have contacted Pacbio's Aaron Wenger and Gary Gelland regarding this matter and they've advice me to post the error on Github if all fails. I've tried with and without thread and chunksize options.

COMMAND: python aligned_bam_to_cpg_scores.py -b {merged.grch37_mapped.bam} -f {reference_grch37.fasta} -o merged.grch37_mapped --pileup_mode count --modsites denovo --thread 24 --chunksize 50000

ERROR: Chunking regions for multiprocessing. Running multiprocessing on 62,786 chunks. 82%|██████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 51330/62786 [17:21:18<3:52:24, 1.22s/it]Traceback (most recent call last): File "/stornext/snfs4/next-gen/scratch/yzhu_u231245/aou_lr_2022/pb_CpG_tools/aligned_bam_to_cpg_scores.py", line 1152, in main() File "/stornext/snfs4/next-gen/scratch/yzhu_u231245/aou_lr_2022/pb_CpG_tools/aligned_bam_to_cpg_scores.py", line 1140, in main bed_results = run_all_pileup_processing(regions_to_process, args.threads) File "/stornext/snfs4/next-gen/scratch/yzhu_u231245/aou_lr_2022/pb_CpG_tools/aligned_bam_to_cpg_scores.py", line 915, in run_all_pileup_processing bed_result = future.result() File "/stornext/snfs5/next-gen/scratch/yzhu_u231245/miniconda3/envs/cpg/lib/python3.9/concurrent/futures/_base.py", line 439, in result return self.__get_result() File "/stornext/snfs5/next-gen/scratch/yzhu_u231245/miniconda3/envs/cpg/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result raise self._exception concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Thank you, Yiming "Ming" Zhu BCM-HGSC

RhettRautsaw commented 2 years ago

Hi @yimingzhu100.

There's another issue open with this problem. I've just posted a comment on there with my attempt to address this issue.

https://github.com/PacificBiosciences/pb-CpG-tools/issues/26#issuecomment-1240821179

ctsa commented 2 years ago

@yimingzhu100 - if you're still having trouble see my comment about an alternate pre-release binary here:

https://github.com/PacificBiosciences/pb-CpG-tools/issues/26#issuecomment-1248757023

ctsa commented 2 years ago

Closing as presumed dup of #26