Closed bschilder closed 2 years ago
bschilder
commented
3 years ago Same issue across multiple nf-core pipelines, but no clear solutions.
With nf-core/atacseq: https://github.com/nf-core/atacseq/issues/54
Same error message documented here within the nf-core/ampliseq pipeline, but no clear solution from what I can tell (problem just disappeared?)
https://github.com/nf-core/ampliseq/issues/102
https://github.com/nf-core/ampliseq/issues/55
NathanSkene
commented
3 years ago Have you tried relaunching to see if it fails the same way second time?
bschilder
commented
3 years ago Rerunning now after deleting all contents of /rds/general/user/bms20/ephemeral/tmp. It's already gotten further than the previous run (currently on MERGED_LIB_BAM), so fingers crossed!
Haven't quite figured out the rhyme or reason to this pipeline working or not, but will be sure to post here if I do.
bschilder
commented
3 years ago ahhh, spoke too soon. Failed with a new error:
Here's the crux of it. Haven't seen this one before, failing on picard due to A fatal error has been detected by the Java Runtime Environment:
command exit status:
134
Command output:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0x7) at pc=0x000014e462f36c8c, pid=720392, tid=720445
#
# JRE version: OpenJDK Runtime Environment (11.0.1) (build 11.0.1-internal+0-adhoc..src)
# Java VM: OpenJDK 64-Bit Server VM (11.0.1-internal+0-adhoc..src, mixed mode, tiered, g1 gc, linux-amd64)
# Problematic frame:
# V [libjvm.so+0x7efc8c] HSpaceCounters::update_used(unsigned long)+0xc
#
# Core dump will be written. Default location: /rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/core.%p
#
# An error report file with more information is saved as:
# /rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/hs_err_pid720392.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
#If you redo the run on our data that previously worked, does that run successfully? Is it actually to do with the design file, or has the HPC just gone haywire?
On Thu, 21 Jan 2021 at 13:31, Brian M. Schilder notifications@github.com wrote:
This email from notifications@github.com originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address.
ahhh, spoke too soon. Failed with a new error:
Here's the crux of it. Haven't seen this one before, failing on picard due to A fatal error has been detected by the Java Runtime Environment:
command exit status:
134
Command output:
#
A fatal error has been detected by the Java Runtime Environment:
#
SIGBUS (0x7) at pc=0x000014e462f36c8c, pid=720392, tid=720445
#
JRE version: OpenJDK Runtime Environment (11.0.1) (build 11.0.1-internal+0-adhoc..src)
Java VM: OpenJDK 64-Bit Server VM (11.0.1-internal+0-adhoc..src, mixed mode, tiered, g1 gc, linux-amd64)
Problematic frame:
V [libjvm.so+0x7efc8c] HSpaceCounters::update_used(unsigned long)+0xc
#
Core dump will be written. Default location: /rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/core.%p
#
An error report file with more information is saved as:
/rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/hs_err_pid720392.log
#
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
#
Full Error
N E X T F L O W ~ version 20.10.0
Launching
nf-core/atacseq[pensive_austin] - revision: 1b3a832db5 [1.2.1]
,--./,-. ___ __ __ __ ___ /,-._.--~'|\ | | / ` / \ |) | } {
| | | _, _/ | \ |__ `-.,-`-,
`._,._,'nf-core/atacseq v1.2.1
Run Name : pensive_austin
Data Type : Paired-End
Design File : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/raw_data/HK5M2BBXY/design_assays.csv
Genome : GRCh37
Fasta File : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/WholeGenomeFasta/genome.fa
GTF File : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.gtf
Gene BED File : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed
BWA Index : s3://ngi-igenomes/igenomes//Homo_sapiens/Ensembl/GRCh37/Sequence/BWAIndex/genome.fa
Blacklist BED : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq/assets/blacklists/GRCh37-blacklist.bed
Mitochondrial Contig : MT
MACS2 Genome Size : 2.7e9
Min Consensus Reps : 1
MACS2 Narrow Peaks : No
MACS2 Broad Cutoff : 0.1
Trim R1 : 0 bp
Trim R2 : 0 bp
Trim 3' R1 : 0 bp
Trim 3' R2 : 0 bp
NextSeq Trim : 0 bp
Fingerprint Bins : 500000
Save Genome Index : No
Max Resources : 128 GB memory, 16 cpus, 10d time per job
Container : singularity - /rds/general/user/bms20/home/atacseq_latest.sif
Output Dir : /rds/general/user/bms20/home/neurogenomics/GitRepos/CUT_n_TAG/processed_data/HK5M2BBXY
Launch Dir : /rds/general/project/neurogenomics-lab/live/GitRepos/CUT_n_TAG
Working Dir : /rds/general/user/bms20/ephemeral/tmp
Script Dir : /rds/general/user/bms20/home/.nextflow/assets/nf-core/atacseq
User : bms20
Config Profile : standard
executor > pbspro (18)
[7d/0cfb28] process > CHECK_DESIGN (design_assays.csv) [100%] 1 of 1 ✔
[ce/1f5797] process > MAKE_TSS_BED (genes.bed) [100%] 1 of 1 ✔
[c3/4b7ed1] process > MAKE_GENOME_FILTER (genome.fa) [100%] 1 of 1 ✔
[fa/8b05a5] process > FASTQC (H3K27me3_R1_T1) [100%] 3 of 3, failed: 1, retries: 1 ✔
[83/13f6ec] process > TRIMGALORE (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[66/5becb3] process > BWA_MEM (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[01/97f88b] process > SORT_BAM (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[3f/366009] process > MERGED_LIB_BAM (H3K27me3_R1) [ 50%] 2 of 4, failed: 2, retries: 2
[- ] process > MERGED_LIB_BAM_FILTER -
[- ] process > MERGED_LIB_BAM_REMOVE_ORPHAN -
[- ] process > MERGED_LIB_PRESEQ -
[- ] process > MERGED_LIB_PICARD_METRICS -
[- ] process > MERGED_LIB_BIGWIG -
[- ] process > MERGED_LIB_PLOTPROFILE -
[- ] process > MERGED_LIB_PLOTFINGERPRINT -
[- ] process > MERGED_LIB_MACS2 -
[- ] process > MERGED_LIB_MACS2_ANNOTATE -
[- ] process > MERGED_LIB_MACS2_QC -
[- ] process > MERGED_LIB_CONSENSUS -
[- ] process > MERGED_LIB_CONSENSUS_ANNOTATE -
[- ] process > MERGED_LIB_CONSENSUS_COUNTS -
[- ] process > MERGED_LIB_CONSENSUS_DESEQ2 -
[- ] process > MERGED_LIB_ATAQV -
[- ] process > MERGED_LIB_ATAQV_MKARV -
[- ] process > MERGED_REP_BAM -
[- ] process > MERGED_REP_BIGWIG -
[- ] process > MERGED_REP_MACS2 -
[- ] process > MERGED_REP_MACS2_ANNOTATE -
[- ] process > MERGED_REP_MACS2_QC -
[- ] process > MERGED_REP_CONSENSUS -
[- ] process > MERGED_REP_CONSENSUS_ANNOTATE -
[- ] process > MERGED_REP_CONSENSUS_COUNTS -
[- ] process > MERGED_REP_CONSENSUS_DESEQ2 -
[- ] process > IGV -
[5c/1848e7] process > get_software_versions [100%] 1 of 1 ✔
[- ] process > MULTIQC -
[7d/f3dc8a] process > output_documentation [100%] 1 of 1 ✔
Error executing process > 'MERGED_LIB_BAM (H3K27ac_R1)'
Caused by:
Process
MERGED_LIB_BAM (H3K27ac_R1)terminated with an error exit status (134)Command executed:
picard -Xmx72g MarkDuplicates \
INPUT=H3K27ac_R1_T1.Lb.sorted.bam \ OUTPUT=H3K27ac_R1.mLb.mkD.sorted.bam \ ASSUME_SORTED=true \ REMOVE_DUPLICATES=false \ METRICS_FILE=H3K27ac_R1.mLb.mkD.MarkDuplicates.metrics.txt \ VALIDATION_STRINGENCY=LENIENT \ TMP_DIR=tmpsamtools index H3K27ac_R1.mLb.mkD.sorted.bam
samtools idxstats H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.idxstats
samtools flagstat H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.flagstat
samtools stats H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.stats
Command exit status:
134
Command output:
#
A fatal error has been detected by the Java Runtime Environment:
#
SIGBUS (0x7) at pc=0x000014e462f36c8c, pid=720392, tid=720445
#
JRE version: OpenJDK Runtime Environment (11.0.1) (build 11.0.1-internal+0-adhoc..src)
Java VM: OpenJDK 64-Bit Server VM (11.0.1-internal+0-adhoc..src, mixed mode, tiered, g1 gc, linux-amd64)
Problematic frame:
V [libjvm.so+0x7efc8c] HSpaceCounters::update_used(unsigned long)+0xc
#
Core dump will be written. Default location: /rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/core.%p
#
An error report file with more information is saved as:
/rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/hs_err_pid720392.log
#
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
#
Command error:
INFO 2021-01-21 13:10:55 MarkDuplicates Read 5,000,000 records. Elapsed time: 00:00:50s. Time for last 1,000,000: 9s. Last read position: 5:134,863,199
INFO 2021-01-21 13:10:55 MarkDuplicates Tracking 18810 as yet unmatched pairs. 377 records in RAM.
INFO 2021-01-21 13:11:04 MarkDuplicates Read 6,000,000 records. Elapsed time: 00:00:59s. Time for last 1,000,000: 9s. Last read position: 6:122,969,223
INFO 2021-01-21 13:11:04 MarkDuplicates Tracking 18984 as yet unmatched pairs. 398 records in RAM.
INFO 2021-01-21 13:11:14 MarkDuplicates Read 7,000,000 records. Elapsed time: 00:01:09s. Time for last 1,000,000: 9s. Last read position: 7:135,215,448
INFO 2021-01-21 13:11:14 MarkDuplicates Tracking 18912 as yet unmatched pairs. 319 records in RAM.
INFO 2021-01-21 13:11:23 MarkDuplicates Read 8,000,000 records. Elapsed time: 00:01:18s. Time for last 1,000,000: 9s. Last read position: 9:68,412,313
INFO 2021-01-21 13:11:23 MarkDuplicates Tracking 18492 as yet unmatched pairs. 1105 records in RAM.
INFO 2021-01-21 13:11:33 MarkDuplicates Read 9,000,000 records. Elapsed time: 00:01:28s. Time for last 1,000,000: 9s. Last read position: 10:88,012,107
INFO 2021-01-21 13:11:33 MarkDuplicates Tracking 17668 as yet unmatched pairs. 666 records in RAM.
INFO 2021-01-21 13:11:42 MarkDuplicates Read 10,000,000 records. Elapsed time: 00:01:37s. Time for last 1,000,000: 9s. Last read position: 11:126,859,165
INFO 2021-01-21 13:11:42 MarkDupli0:26 MarkDuplicates Tracking 5216 as yet unmatched pairs. 1336 records in RAM.
INFO 2021-01-21 13:10:36 MarkDuplicates Read 3,000,000 records. Elapsed time: 00:00:31s. Time for last 1,000,000: 9s. Last read position: 2:236,961,048
INFO 2021-01-21 13:10:36 MarkDuplicates Tracking 16352 as yet unmatched pairs. 77 records in RAM.
INFO 2021-01-21 13:10:46 MarkDuplicates Read 4,000,000 records. Elapsed time: 00:00:40s. Time for last 1,000,000: 9s. Last read position: 4:8,331,727
INFO 2021-01-21 13:10:46 MarkDuplicates Tracking 17242 as yet unmatched pairs. 1078 records in RAM.
INFO 2021-01-21 13:10:55 MarkDuplicates Read 5,000,000 records. Elapsed time: 00:00:50s. Time for last 1,000,000: 9s. Last read position: 5:134,863,199
INFO 2021-01-21 13:10:55 MarkDuplicates Tracking 18810 as yet unmatched pairs. 377 records in RAM.
INFO 2021-01-21 13:11:04 MarkDuplicates Read 6,000,000 records. Elapsed time: 00:00:59s. Time for last 1,000,000: 9s. Last read position: 6:122,969,223
INFO 2021-01-21 13:11:04 MarkDuplicates Tracking 18984 as yet unmatched pairs. 398 records in RAM.
INFO 2021-01-21 13:11:14 MarkDuplicates Read 7,000,000 records. Elapsed time: 00:01:09s. Time for last 1,000,000: 9s. Last read position: 7:135,215,448
INFO 2021-01-21 13:11:14 MarkDuplicates Tracking 18912 as yet unmatched pairs. 319 records in RAM.
INFO 2021-01-21 13:11:23 MarkDuplicates Read 8,000,000 records. Elapsed time: 00:01:18s. Time for last 1,000,000: 9s. Last read position: 9:68,412,313
INFO 2021-01-21 13:11:23 MarkDuplicates Tracking 18492 as yet unmatched pairs. 1105 records in RAM.
INFO 2021-01-21 13:11:33 MarkDuplicates Read 9,000,000 records. Elapsed time: 00:01:28s. Time for last 1,000,000: 9s. Last read position: 10:88,012,107
executor > pbspro (18)
[7d/0cfb28] process > CHECK_DESIGN (design_assays.csv) [100%] 1 of 1 ✔
[ce/1f5797] process > MAKE_TSS_BED (genes.bed) [100%] 1 of 1 ✔
[c3/4b7ed1] process > MAKE_GENOME_FILTER (genome.fa) [100%] 1 of 1 ✔
[fa/8b05a5] process > FASTQC (H3K27me3_R1_T1) [100%] 3 of 3, failed: 1, retries: 1 ✔
[83/13f6ec] process > TRIMGALORE (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[66/5becb3] process > BWA_MEM (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[01/97f88b] process > SORT_BAM (H3K27me3_R1_T1) [100%] 2 of 2 ✔
[c6/cbf48d] process > MERGED_LIB_BAM (H3K27ac_R1) [ 75%] 3 of 4, failed: 3, retries: 2
[- ] process > MERGED_LIB_BAM_FILTER -
[- ] process > MERGED_LIB_BAM_REMOVE_ORPHAN -
[- ] process > MERGED_LIB_PRESEQ -
[- ] process > MERGED_LIB_PICARD_METRICS -
[- ] process > MERGED_LIB_BIGWIG -
[- ] process > MERGED_LIB_PLOTPROFILE -
[- ] process > MERGED_LIB_PLOTFINGERPRINT -
[- ] process > MERGED_LIB_MACS2 -
[- ] process > MERGED_LIB_MACS2_ANNOTATE -
[- ] process > MERGED_LIB_MACS2_QC -
[- ] process > MERGED_LIB_CONSENSUS -
[- ] process > MERGED_LIB_CONSENSUS_ANNOTATE -
[- ] process > MERGED_LIB_CONSENSUS_COUNTS -
[- ] process > MERGED_LIB_CONSENSUS_DESEQ2 -
[- ] process > MERGED_LIB_ATAQV -
[- ] process > MERGED_LIB_ATAQV_MKARV -
[- ] process > MERGED_REP_BAM -
[- ] process > MERGED_REP_BIGWIG -
[- ] process > MERGED_REP_MACS2 -
[- ] process > MERGED_REP_MACS2_ANNOTATE -
[- ] process > MERGED_REP_MACS2_QC -
[- ] process > MERGED_REP_CONSENSUS -
[- ] process > MERGED_REP_CONSENSUS_ANNOTATE -
[- ] process > MERGED_REP_CONSENSUS_COUNTS -
[- ] process > MERGED_REP_CONSENSUS_DESEQ2 -
[- ] process > IGV -
[5c/1848e7] process > get_software_versions [100%] 1 of 1 ✔
[- ] process > MULTIQC -
[7d/f3dc8a] process > output_documentation [100%] 1 of 1 ✔
Error executing process > 'MERGED_LIB_BAM (H3K27ac_R1)'
Caused by:
Process
MERGED_LIB_BAM (H3K27ac_R1)terminated with an error exit status (134)Command executed:
picard -Xmx72g MarkDuplicates \
INPUT=H3K27ac_R1_T1.Lb.sorted.bam \ OUTPUT=H3K27ac_R1.mLb.mkD.sorted.bam \ ASSUME_SORTED=true \ REMOVE_DUPLICATES=false \ METRICS_FILE=H3K27ac_R1.mLb.mkD.MarkDuplicates.metrics.txt \ VALIDATION_STRINGENCY=LENIENT \ TMP_DIR=tmpsamtools index H3K27ac_R1.mLb.mkD.sorted.bam
samtools idxstats H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.idxstats
samtools flagstat H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.flagstat
samtools stats H3K27ac_R1.mLb.mkD.sorted.bam > H3K27ac_R1.mLb.mkD.sorted.bam.stats
Command exit status:
134
Command output:
#
A fatal error has been detected by the Java Runtime Environment:
#
SIGBUS (0x7) at pc=0x000014e462f36c8c, pid=720392, tid=720445
#
JRE version: OpenJDK Runtime Environment (11.0.1) (build 11.0.1-internal+0-adhoc..src)
Java VM: OpenJDK 64-Bit Server VM (11.0.1-internal+0-adhoc..src, mixed mode, tiered, g1 gc, linux-amd64)
Problematic frame:
V [libjvm.so+0x7efc8c] HSpaceCounters::update_used(unsigned long)+0xc
#
Core dump will be written. Default location: /rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/core.%p
#
An error report file with more information is saved as:
/rds/general/ephemeral/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4/hs_err_pid720392.log
#
If you would like to submit a bug report, please visit:
http://bugreport.java.com/bugreport/crash.jsp
#
Command error:
INFO 2021-01-21 13:10:55 MarkDuplicates Read 5,000,000 records. Elapsed time: 00:00:50s. Time for last 1,000,000: 9s. Last read position: 5:134,863,199
INFO 2021-01-21 13:10:55 MarkDuplicates Tracking 18810 as yet unmatched pairs. 377 records in RAM.
INFO 2021-01-21 13:11:04 MarkDuplicates Read 6,000,000 records. Elapsed time: 00:00:59s. Time for last 1,000,000: 9s. Last read position: 6:122,969,223
INFO 2021-01-21 13:11:04 MarkDuplicates Tracking 18984 as yet unmatched pairs. 398 records in RAM.
INFO 2021-01-21 13:11:14 MarkDuplicates Read 7,000,000 records. Elapsed time: 00:01:09s. Time for last 1,000,000: 9s. Last read position: 7:135,215,448
INFO 2021-01-21 13:11:14 MarkDuplicates Tracking 18912 as yet unmatched pairs. 319 records in RAM.
INFO 2021-01-21 13:11:23 MarkDuplicates Read 8,000,000 records. Elapsed time: 00:01:18s. Time for last 1,000,000: 9s. Last read position: 9:68,412,313
INFO 2021-01-21 13:11:23 MarkDuplicates Tracking 18492 as yet unmatched pairs. 1105 records in RAM.
INFO 2021-01-21 13:11:33 MarkDuplicates Read 9,000,000 records. Elapsed time: 00:01:28s. Time for last 1,000,000: 9s. Last read position: 10:88,012,107
INFO 2021-01-21 13:11:33 MarkDuplicates Tracking 17668 as yet unmatched pairs. 666 records in RAM.
INFO 2021-01-21 13:11:42 MarkDuplicates Read 10,000,000 records. Elapsed time: 00:01:37s. Time for last 1,000,000: 9s. Last read position: 11:126,859,165
INFO 2021-01-21 13:11:42 MarkDupli0:26 MarkDuplicates Tracking 5216 as yet unmatched pairs. 1336 records in RAM.
INFO 2021-01-21 13:10:36 MarkDuplicates Read 3,000,000 records. Elapsed time: 00:00:31s. Time for last 1,000,000: 9s. Last read position: 2:236,961,048
INFO 2021-01-21 13:10:36 MarkDuplicates Tracking 16352 as yet unmatched pairs. 77 records in RAM.
INFO 2021-01-21 13:10:46 MarkDuplicates Read 4,000,000 records. Elapsed time: 00:00:40s. Time for last 1,000,000: 9s. Last read position: 4:8,331,727
INFO 2021-01-21 13:10:46 MarkDuplicates Tracking 17242 as yet unmatched pairs. 1078 records in RAM.
INFO 2021-01-21 13:10:55 MarkDuplicates Read 5,000,000 records. Elapsed time: 00:00:50s. Time for last 1,000,000: 9s. Last read position: 5:134,863,199
INFO 2021-01-21 13:10:55 MarkDuplicates Tracking 18810 as yet unmatched pairs. 377 records in RAM.
INFO 2021-01-21 13:11:04 MarkDuplicates Read 6,000,000 records. Elapsed time: 00:00:59s. Time for last 1,000,000: 9s. Last read position: 6:122,969,223
INFO 2021-01-21 13:11:04 MarkDuplicates Tracking 18984 as yet unmatched pairs. 398 records in RAM.
INFO 2021-01-21 13:11:14 MarkDuplicates Read 7,000,000 records. Elapsed time: 00:01:09s. Time for last 1,000,000: 9s. Last read position: 7:135,215,448
INFO 2021-01-21 13:11:14 MarkDuplicates Tracking 18912 as yet unmatched pairs. 319 records in RAM.
INFO 2021-01-21 13:11:23 MarkDuplicates Read 8,000,000 records. Elapsed time: 00:01:18s. Time for last 1,000,000: 9s. Last read position: 9:68,412,313
INFO 2021-01-21 13:11:23 MarkDuplicates Tracking 18492 as yet unmatched pairs. 1105 records in RAM.
INFO 2021-01-21 13:11:33 MarkDuplicates Read 9,000,000 records. Elapsed time: 00:01:28s. Time for last 1,000,000: 9s. Last read position: 10:88,012,107
INFO 2021-01-21 13:11:33 MarkDuplicates Tracking 17668 as yet unmatched pairs. 666 records in RAM.
INFO 2021-01-21 13:11:42 MarkDuplicates Read 10,000,000 records. Elapsed time: 00:01:37s. Time for last 1,000,000: 9s. Last read position: 11:126,859,165
INFO 2021-01-21 13:11:42 MarkDuplicates Tracking 16628 as yet unmatched pairs. 80 records in RAM.
INFO 2021-01-21 13:11:52 MarkDuplicates Read 11,000,000 records. Elapsed time: 00:01:47s. Time for last 1,000,000: 9s. Last read position: 14:35,152,111
INFO 2021-01-21 13:11:52 MarkDuplicates Tracking 15078 as yet unmatched pairs. 830 records in RAM.
INFO 2021-01-21 13:12:01 MarkDuplicates Read 12,000,000 records. Elapsed time: 00:01:56s. Time for last 1,000,000: 9s. Last read position: 16:15,004,892
INFO 2021-01-21 13:12:01 MarkDuplicates Tracking 13436 as yet unmatched pairs. 2365 records in RAM.
INFO 2021-01-21 13:12:11 MarkDuplicates Read 13,000,000 records. Elapsed time: 00:02:06s. Time for last 1,000,000: 9s. Last read position: 17:61,435,332
INFO 2021-01-21 13:12:11 MarkDuplicates Tracking 10916 as yet unmatched pairs. 575 records in RAM.
INFO 2021-01-21 13:12:20 MarkDuplicates Read 14,000,000 records. Elapsed time: 00:02:15s. Time for last 1,000,000: 9s. Last read position: 19:44,188,594
INFO 2021-01-21 13:12:20 MarkDuplicates Tracking 8658 as yet unmatched pairs. 627 records in RAM.
INFO 2021-01-21 13:12:30 MarkDuplicates Read 15,000,000 records. Elapsed time: 00:02:25s. Time for last 1,000,000: 9s. Last read position: 22:21,368,723
INFO 2021-01-21 13:12:30 MarkDuplicates Tracking 5610 as yet unmatched pairs. 1024 records in RAM.
INFO 2021-01-21 13:12:38 MarkDuplicates Read 15803318 records. 0 pairs never matched.
INFO 2021-01-21 13:12:38 MarkDuplicates After buildSortedReadEndLists freeMemory: 4648679064; totalMemory: 11056185344; maxMemory: 77309411328
INFO 2021-01-21 13:12:38 MarkDuplicates Will retain up to 2147483642 duplicate indices before spilling to disk.
INFO 2021-01-21 13:12:46 MarkDuplicates Traversing read pair information and detecting duplicates.
INFO 2021-01-21 13:12:52 MarkDuplicates Traversing fragment information and detecting duplicates.
INFO 2021-01-21 13:12:54 MarkDuplicates Sorting list of duplicate records.
INFO 2021-01-21 13:12:55 MarkDuplicates After generateDuplicateIndexes freeMemory: 11725184352; totalMemory: 28923920384; maxMemory: 77309411328
INFO 2021-01-21 13:12:55 MarkDuplicates Marking 8808507 records as duplicates.
INFO 2021-01-21 13:12:55 MarkDuplicates Found 32218 optical duplicate clusters.
INFO 2021-01-21 13:12:55 MarkDuplicates Reads are assumed to be ordered by: coordinate
INFO 2021-01-21 13:17:29 MarkDuplicates Written 10,000,000 records. Elapsed time: 00:04:33s. Time for last 10,000,000: 273s. Last read position: 11:126,859,165
/opt/conda/envs/nf-core-atacseq-1.2.1/bin/picard: line 66: 720392 Aborted (core dumped) /opt/conda/envs/nf-core-atacseq-1.2.1/bin/java -Xmx72g -jar /opt/conda/envs/nf-core-atacseq-1.2.1/share/picard-2.23.1-0/picard.jar MarkDuplicates "INPUT=H3K27ac_R1_T1.Lb.sorted.bam" "OUTPUT=H3K27ac_R1.mLb.mkD.sorted.bam" "ASSUME_SORTED=true" "REMOVE_DUPLICATES=false" "METRICS_FILE=H3K27ac_R1.mLb.mkD.MarkDuplicates.metrics.txt" "VALIDATION_STRINGENCY=LENIENT" "TMP_DIR=tmp"
Work dir:
/rds/general/user/bms20/ephemeral/tmp/c6/cbf48df0c06ecc5b61e18ff43461a4
Tip: you can replicate the issue by changing to the process work dir and entering the command
bash .command.run-[nf-core/atacseq] Pipeline completed with errors-
WARN: Killing pending tasks (1)
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/neurogenomics/CUT_n_TAG/issues/6#issuecomment-764644124, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH5ZPE44YYD3QMARQ7CEIILS3AUENANCNFSM4WMSGIUA .
bschilder
commented
3 years ago It could either or both, really not sure at this point. Here's the full error log: https://github.com/neurogenomics/CUT_n_TAG/blob/main/error_log.txt
I've just read through the whole thing and can't really make any sense of it.
Previously, I accidentally mis-specified my design file so that it merged H3K27me3 and H3K27ac data, such that both were placed in the group "control" (
design_noindex2.csv).This time, I'm rerunning with a new design file to separate the assays (
design_assays.csv).However, now I'm encountering a new error, which may or may not be related to the change in the design file:
Some steps run to completion, but others fail entirely (e.g. bwa).
Error
-bash-4.2$ bash run_atacseq.sh