search

ENCODE-DCC / chip-seq-pipeline2

ENCODE ChIP-seq pipeline
MIT License
241 stars 123 forks source link

spp step without output for a long time #113

Closed yifeisun03 closed 4 years ago

yifeisun03 commented 4 years ago

Hello:

Do you know how long does spp peak calling normally takes?

The pipeline still seems to be running( LSF mem use around 50G), and the only steps not "all done" are the call peaks steps(spp), I see all the output directly in my outdir( call-call-peak, call-call-peak-ppr1,etc). It's already been 24h since this step started, but there's no output at all in the "execution",stderr empty, tep empty. Is this normal, or should I be seeing some intermediate files being made? Thanks!

stdout(only for spp step) as below [2019-12-09 11:28:32,489 INFO] ['/hpc/users/suny04/.conda/envs/encode-chip-seq-pipeline/bin/encode_task_spp.py', '/sc/orga/projects/walshm02a/Yifei/bsub/chip-caper/chip/3ec5a3d4-32cc-4a79-877b-6fa71b6e09\ c4/call-call_peak_pooled/inputs/32247729/A15_CKDL190138594-1a-AK445_HMFLWDSXX_L1_1.merged.nodup.pooled.tagAlign.gz', '/sc/orga/projects/walshm02a/Yifei/bsub/chip-caper/chip/3ec5a3d4-32cc-4a79-877b-6fa71b\ 6e09c4/call-call_peak_pooled/inputs/-23361139/A20_CKDL190138594-1a-AK1668_HMFLWDSXX_L1_1.merged.nodup.pooled.tagAlign.gz', '--fraglen', '105', '--cap-num-peak', '300000', '--nth', '2'] [2019-12-09 11:28:32,489 INFO] Initializing and making output directory... [2019-12-09 11:28:32,489 INFO] Calling peaks with spp... [2019-12-09 11:28:32,518 INFO] run_shell_cmd: PID=337446, PGID=337446, CMD=Rscript --max-ppsize=500000 $(which run_spp.R) -c=/sc/orga/projects/walshm02a/Yifei/bsub/chip-caper/chip/3ec5a3d4-32cc-4a79-877b\ -6fa71b6e09c4/call-call_peak_pooled/inputs/32247729/A15_CKDL190138594-1a-AK445_HMFLWDSXX_L1_1.merged.nodup.pooled.tagAlign.gz -i=/sc/orga/projects/walshm02a/Yifei/bsub/chip-caper/chip/3ec5a3d4-32cc-4a79-\ 877b-6fa71b6e09c4/call-call_peak_pooled/inputs/-23361139/A20_CKDL190138594-1a-AK1668_HMFLWDSXX_L1_1.merged.nodup.pooled.tagAlign.gz -npeak=300000 -odir=/sc/orga/projects/walshm02a/Yifei/bsub/chip-caper/c\ hip/3ec5a3d4-32cc-4a79-877b-6fa71b6e09c4/call-call_peak_pooled/execution -speak=105 -savr=A15_CKDL190138594-1a-AK445_HMFLWDSXX_L1_1.merged.nodup.pooled_x_A20_CKDL190138594-1a-AK1668_HMFLWDSXX_L1_1.merged\ .nodup.pooled.300K.regionPeak.gz.tmp -rf

OS/Platform

leepc12 commented 4 years ago

SPP takes the longest time among all tasks. It depends on the number of CPU cores but it can take > 1 day for some large samples.