Open sunta3iouxos opened 2 months ago
Hi Theodoros,
I'm sorry to hear that plot fingerpring fails for you. Why do you think it should take more than 6 hours? Ideally, it should be done within minutes. Perhaps increasing the memory resource would help it run faster on your system?
The zombie jobs are not unusual with anything written in python.
Best wishes, Katarzyna
Hi Katarzyna, this is the error I get. Notice that the provided memory is: mem_mb=115345, disk_mb=115345
---- This analysis has been done using snakePipes version 2.8.1 ----
Sample sheet found and header is ok!
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control
warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 24
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=115345, disk_mb=115345
Select jobs to execute...
[Tue Apr 16 09:32:35 2024]
rule plotFingerprint:
input: filtered_bam/A006200376_218434_S7_L001.filtered.bam, filtered_bam/A006200376_218437_S8_L001.filtered.bam, filtered_bam/A006200376_218439_S9_L001.filtered.bam, filtered_bam/A006200376_218442_S10_L001.filtered.bam, filtered_bam/A006200376_218444_S11_L001.filtered.bam, filtered_bam/A006200376_218446_S12_L001.filtered.bam, filtered_bam/A006200376_218448_S13_L001.filtered.bam, filtered_bam/A006200376_218450_S14_L001.filtered.bam, filtered_bam/A006200376_218452_S15_L001.filtered.bam, filtered_bam/A006200376_218454_S16_L001.filtered.bam, filtered_bam/A006200376_218456_S17_L001.filtered.bam, filtered_bam/A006200376_218458_S18_L001.filtered.bam, filtered_bam/A006200376_218460_S19_L001.filtered.bam, filtered_bam/A006200376_218462_S20_L001.filtered.bam, filtered_bam/A006200376_218464_S21_L001.filtered.bam, filtered_bam/A006200376_218466_S22_L001.filtered.bam, filtered_bam/A006200376_218468_S23_L001.filtered.bam, filtered_bam/A006200376_218470_S24_L001.filtered.bam, filtered_bam/A006200376_218472_S25_L001.filtered.bam, filtered_bam/A006200376_218474_S26_L001.filtered.bam, filtered_bam/A006200376_218434_S7_L001.filtered.bam.bai, filtered_bam/A006200376_218437_S8_L001.filtered.bam.bai, filtered_bam/A006200376_218439_S9_L001.filtered.bam.bai, filtered_bam/A006200376_218442_S10_L001.filtered.bam.bai, filtered_bam/A006200376_218444_S11_L001.filtered.bam.bai, filtered_bam/A006200376_218446_S12_L001.filtered.bam.bai, filtered_bam/A006200376_218448_S13_L001.filtered.bam.bai, filtered_bam/A006200376_218450_S14_L001.filtered.bam.bai, filtered_bam/A006200376_218452_S15_L001.filtered.bam.bai, filtered_bam/A006200376_218454_S16_L001.filtered.bam.bai, filtered_bam/A006200376_218456_S17_L001.filtered.bam.bai, filtered_bam/A006200376_218458_S18_L001.filtered.bam.bai, filtered_bam/A006200376_218460_S19_L001.filtered.bam.bai, filtered_bam/A006200376_218462_S20_L001.filtered.bam.bai, filtered_bam/A006200376_218464_S21_L001.filtered.bam.bai, filtered_bam/A006200376_218466_S22_L001.filtered.bam.bai, filtered_bam/A006200376_218468_S23_L001.filtered.bam.bai, filtered_bam/A006200376_218470_S24_L001.filtered.bam.bai, filtered_bam/A006200376_218472_S25_L001.filtered.bam.bai, filtered_bam/A006200376_218474_S26_L001.filtered.bam.bai
output: deepTools_ChIP/plotFingerprint/plotFingerprint.metrics.txt
log: deepTools_ChIP/logs/plotFingerprint.out, deepTools_ChIP/logs/plotFingerprint.err
jobid: 0
benchmark: deepTools_ChIP/.benchmark/plotFingerprint.benchmark
reason: Missing output files: deepTools_ChIP/plotFingerprint/plotFingerprint.metrics.txt, deepTools_ChIP/.benchmark/plotFingerprint.benchmark
threads: 24
resources: mem_mb=115345, disk_mb=115345, tmpdir=/scratch/tgeorgom_temp
Activating conda environment: ../../mamba/snakepipes_devel/envs/c21237e8c0954c1620210ed0f7ba2e63_
slurmstepd: error: *** JOB 18773077 ON cheops11804 CANCELLED AT 2024-04-16T15:31:45 DUE TO TIME LIMIT ***
there is also this thing that I do not know if it is relevant:
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control
warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")
345
Hi,
am I reading this correctly, that rule plotFingerprint is given 100Gb memory? That's a lot. I believe, by default, it should get 1Gb. Did you modify the memory resource yourself?
Best,
Katarzyna
there is also this thing that I do not know if it is relevant:
/scratch/tgeorgom/mamba/snakepipes_devel/lib/python3.11/site-packages/snakePipes/workflows/ChIP-seq/internals.snakefile:145: UserWarning: 16 out of 16 have a matching control warnings.warn( str(len(chip_samples_w_ctrl)) + " out of " + str(len(chip_samples)) + " have a matching control ")
That's not related. It's just a warning message if control samples were found in the chipdict yaml file or not. Saves a lot of headache with some cutNx experiments.
345
Hi,
am I reading this correctly, that rule plotFingerprint is given 100Gb memory? That's a lot. I believe, by default, it should get 1Gb. Did you modify the memory resource yourself?
Best,
Katarzyna
regarding the memory, I increased it just in case. This what I got from another run of the same bam files, BUT in this case I used mm38 instead of mm39.
s h:m:s max_rss max_vms max_uss max_pss io_in io_out mean_load cpu_time
3611.4827 1:00:11 4608.45 88808.61 2212.22 2299.83 0.00 0.00 1649.02 357.03
/scratch/tgeorgom/PAV11/bamGRnCm38/deepTools_ChIP/.benchmark/plotFingerprint.benchmark (END)
with the following settings from lib/python3.11/site-packages/snakePipes/shared/cluster.yaml: plotFingerprint: memory: 3G and from lib/python3.11/site-packages/snakePipes/shared/defaults.yaml max_thread: 24
Hi all, How can I increase the wall time for plotfinger from 6 to 24hrs? it constantly fails. and also does not properly shutdowns the commands. running the squeue there is no job, but running htop, I can see the jobs still lingering and have to mannualy kill them.
Best Theodoros