search

epi2me-labs / wf-pore-c

Other
34 stars 9 forks source link

Process POREC:merge_pairs (1) terminated with an error exit status (1) #34

Closed SwenDiepstraten closed 9 months ago

SwenDiepstraten commented 1 year ago

Operating System

Linux

Other Linux

No response

Workflow Version

v0.1.1-g5f8abc1

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

nextflow run wf-pore-c --cutter 'NlaIII' --ref 'assembly.fasta' --hi_c --threads 32 --out_dir 'output' -profile singularity -c 'pore-c_config.config' --fastq 'combined_pore-c.fastq.gz'

Workflow Execution - CLI Execution Profile

singularity

What happened?

I got an error regarding the merge_pairs. This only happens after the pipeline is running after a long time.

Relevant log output

executor > local (3834)
[d0/9a7cb2] process > POREC:fastcat (1) [100%] 1 of 1 ✔
[64/ddc92b] process > POREC:index_ubam (1) [100%] 1 of 1 ✔
[8d/fc3aec] process > POREC:prepare_genome:index_... [100%] 1 of 1 ✔
[2b/b907ee] process > POREC:prepare_genome:index_... [100%] 1 of 1 ✔
[39/92da21] process > POREC:digest_align_annotate... [100%] 1911 of 1911 ✔
[- ] process > POREC:haplotag_alignments -
[5c/b4568d] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1
[d2/2a86ee] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1
[a2/08f336] process > POREC:create_restriction_be... [100%] 1 of 1 ✔
[32/ed1814] process > POREC:to_pairs_file (1911) [100%] 1911 of 1911 ✔
[87/221b10] process > POREC:merge_pairs (1) [ 0%] 0 of 1
[89/808b42] process > POREC:merge_pairs_stats (1) [ 0%] 0 of 1
[- ] process > POREC:pair_stats_report -
[d1/6bc236] process > POREC:getVersions [100%] 1 of 1 ✔
[cd/4df9fc] process > POREC:getParams [100%] 1 of 1 ✔
[d2/9014bc] process > POREC:makeReport (1) [100%] 1 of 1 ✔
[- ] process > POREC:prepare_hic -
ERROR ~ Error executing process > 'POREC:merge_pairs (1)'

Caused by:
Process POREC:merge_pairs (1) terminated with an error exit status (1)

Command executed:

pairtools merge -o "null.pairs.gz" --concatenate to_merge/src*.pairs.gz

Command exit status:
1

Command output:
(empty)

Command error:
INFO: Converting SIF file to temporary sandbox...
WARNING: underlay of /etc/localtime required more than 50 (78) bind mounts
Traceback (most recent call last):
File "/home/epi2melabs/conda/bin/pairtools", line 11, in
sys.exit(cli())
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 134, in merge
merge_py(
File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 254, in merge_py
subprocess.check_call(command, shell=True, stdout=outstream)
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 359, in check_call
retcode = call(*popenargs, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 340, in call
with Popen(*popenargs, **kwargs) as p:
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 858, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 1720, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: '/bin/sh'
INFO: Cleaning up image...

Work dir:
/lustre/nobackup/WUR/ABGC/dieps003/9.4/work/87/221b100bc0e03aaaff4e37c94c62d3

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

executor > local (3834)
[d0/9a7cb2] process > POREC:fastcat (1) [100%] 1 of 1 ✔
[64/ddc92b] process > POREC:index_ubam (1) [100%] 1 of 1 ✔
[8d/fc3aec] process > POREC:prepare_genome:index_... [100%] 1 of 1 ✔
[2b/b907ee] process > POREC:prepare_genome:index_... [100%] 1 of 1 ✔
[39/92da21] process > POREC:digest_align_annotate... [100%] 1911 of 1911 ✔
[- ] process > POREC:haplotag_alignments -
[5c/b4568d] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1
[d2/2a86ee] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1
[a2/08f336] process > POREC:create_restriction_be... [100%] 1 of 1 ✔
[32/ed1814] process > POREC:to_pairs_file (1911) [100%] 1911 of 1911 ✔
[87/221b10] process > POREC:merge_pairs (1) [100%] 1 of 1, failed: 1 ✘
[89/808b42] process > POREC:merge_pairs_stats (1) [ 0%] 0 of 1
[- ] process > POREC:pair_stats_report -
[d1/6bc236] process > POREC:getVersions [100%] 1 of 1 ✔
[cd/4df9fc] process > POREC:getParams [100%] 1 of 1 ✔
[d2/9014bc] process > POREC:makeReport (1) [100%] 1 of 1 ✔
[- ] process > POREC:prepare_hic -
ERROR ~ Error executing process > 'POREC:merge_pairs (1)'

Caused by:
Process POREC:merge_pairs (1) terminated with an error exit status (1)

Command executed:

pairtools merge -o "null.pairs.gz" --concatenate to_merge/src*.pairs.gz

Command exit status:
1

Command output:
(empty)

Command error:
INFO: Converting SIF file to temporary sandbox...
WARNING: underlay of /etc/localtime required more than 50 (78) bind mounts
Traceback (most recent call last):
File "/home/epi2melabs/conda/bin/pairtools", line 11, in
sys.exit(cli())
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 134, in merge
merge_py(
File "/home/epi2melabs/conda/lib/python3.8/site-packages/pairtools/cli/merge.py", line 254, in merge_py
subprocess.check_call(command, shell=True, stdout=outstream)
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 359, in check_call
retcode = call(*popenargs, **kwargs)
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 340, in call
with Popen(*popenargs, **kwargs) as p:
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 858, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/home/epi2melabs/conda/lib/python3.8/subprocess.py", line 1720, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: '/bin/sh'
INFO: Cleaning up image...

Work dir:
/lustre/nobackup/WUR/ABGC/dieps003/9.4/work/87/221b100bc0e03aaaff4e37c94c62d3

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details
WARN: Killing running tasks (3)

Application activity log entry

No response

sarahjeeeze commented 1 year ago

Hi, thanks for this feedback. I will investigate this, it looks like it is breaking because we are providing too many files for pairtools merge to merge. Out of interest what chunk size did you use? If you used the default then in the meantime whilst we investigate it may be worth increasing the chunk size to 100,000 and seeing if that works.

SwenDiepstraten commented 1 year ago

I have indeed used the default chunk size, will try using chunk size of 100,000

SwenDiepstraten commented 1 year ago

I still have an error, but it is a different one:

executor > local (772) [e3/10baff] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [86/faae18] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [3f/c861e6] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [7e/874345] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [65/25014c] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [11/6b88c0] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [72/37bc46] process > POREC:to_pairs_file (285) [ 52%] 253 of 478 [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [a1/78ea56] process > POREC:getVersions [100%] 1 of 1 ✔ [50/4ad7a7] process > POREC:getParams [100%] 1 of 1 ✔ [18/3f078e] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [- ] process > POREC:prepare_hic - ERROR ~ Error executing process > 'POREC:to_pairs_file (263)'

Caused by: Process POREC:to_pairs_file (263) terminated with an error exit status (255)

Command executed:

pairtools parse2 --output-stats "null.stats.txt" -c "fasta.fai" --single-end --readid-transform 'readID.split(":")[0]' --drop-sam --drop-seq --expand --add-pair-index --add-columns mapq,pos5,pos3,cigar,read_len,matched_bp,algn_ref_span,algn_read_span,dist_to_5,dist_to_3,mismatches "monomers.mm2.ns.bam" > extract_pairs.tmp pairtools restrict -f "fragments.bed" -o "null.pairs.gz" extract_pairs.tmp rm -rf extract_pairs.tmp

Command exit status: 255

Command output: (empty)

Command error: INFO: Converting SIF file to temporary sandbox... FATAL: while extracting /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/singularity/ontresearch-wf-pore-c-sha4071ac979d82e19ec935f2afab6eb2495d71d570.img: root filesystem extraction failed: extract command failed: WARNING: passwd file doesn't exist in container, not updating WARNING: group file doesn't exist in container, not updating WARNING: Skipping mount /etc/hosts [binds]: /etc/hosts doesn't exist in container WARNING: Skipping mount /etc/localtime [binds]: /etc/localtime doesn't exist in container WARNING: Skipping mount proc [kernel]: /proc doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/tmp [tmp]: /tmp doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/var/tmp [tmp]: /var/tmp doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container FATAL ERROR:Failed to create thread : exit status 1

Work dir: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/8a/625cca82e61e2a69fd89d3ea5ba7ff

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

-- Check '.nextflow.log' file for details

executor > local (772) [e3/10baff] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [86/faae18] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [3f/c861e6] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [7e/874345] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [65/25014c] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [11/6b88c0] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [6f/8e0b2f] process > POREC:to_pairs_file (271) [ 53%] 255 of 478, faile... [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [a1/78ea56] process > POREC:getVersions [100%] 1 of 1 ✔ [50/4ad7a7] process > POREC:getParams [100%] 1 of 1 ✔ [18/3f078e] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [- ] process > POREC:prepare_hic - ERROR ~ Error executing process > 'POREC:to_pairs_file (263)'

Caused by: Process POREC:to_pairs_file (263) terminated with an error exit status (255)

Command executed:

pairtools parse2 --output-stats "null.stats.txt" -c "fasta.fai" --single-end --readid-transform 'readID.split(":")[0]' --drop-sam --drop-seq --expand --add-pair-index --add-columns mapq,pos5,pos3,cigar,read_len,matched_bp,algn_ref_span,algn_read_span,dist_to_5,dist_to_3,mismatches "monomers.mm2.ns.bam" > extract_pairs.tmp pairtools restrict -f "fragments.bed" -o "null.pairs.gz" extract_pairs.tmp rm -rf extract_pairs.tmp

Command exit status: 255

Command output: (empty)

Command error: INFO: Converting SIF file to temporary sandbox... FATAL: while extracting /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/singularity/ontresearch-wf-pore-c-sha4071ac979d82e19ec935f2afab6eb2495d71d570.img: root filesystem extraction failed: extract command failed: WARNING: passwd file doesn't exist in container, not updating WARNING: group file doesn't exist in container, not updating WARNING: Skipping mount /etc/hosts [binds]: /etc/hosts doesn't exist in container WARNING: Skipping mount /etc/localtime [binds]: /etc/localtime doesn't exist in container WARNING: Skipping mount proc [kernel]: /proc doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/tmp [tmp]: /tmp doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/var/tmp [tmp]: /var/tmp doesn't exist in container WARNING: Skipping mount /home/WUR/dieps003/mambaforge/envs/nextflow/var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container FATAL ERROR:Failed to create thread : exit status 1

Work dir: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/8a/625cca82e61e2a69fd89d3ea5ba7ff

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

-- Check '.nextflow.log' file for details WARN: Killing running tasks (31)

sarahjeeeze commented 1 year ago

Hi, this looks like an error with your environment or singularity. It is probably worth contacting your system administrator with the error or maybe try running the workflow with docker.

SwenDiepstraten commented 1 year ago

I am running from a fresh mamba environment, I do not know if that could be a problem. unfortunately our HPC is not able to run docker, so I do have to run with singularity

sarahjeeeze commented 1 year ago

Does the error occur at the same place each time you run the workflow? Or is it an intermittent issue?

SwenDiepstraten commented 1 year ago

No it does differ over different runs

sarahjeeeze commented 1 year ago

Oh I suggest not setting the threads parameter to 32, you can leave it as the default of 4. This is the number of threads used by each process - If you want to increase the total number available for the workflow set it in the config cpus. This might be causing the error.

SwenDiepstraten commented 1 year ago

This did improve how far the program got, but I am still encountering an error: executor > local (970) [dc/ad9291] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [6f/4ec9fd] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [b1/1f38fc] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d0/65c656] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [11/301a9e] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [19/c08853] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1 [9d/8872ca] process > POREC:merge_namesorted_bams... [100%] 1 of 1 ✔ [aa/53ff40] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [f8/7018cc] process > POREC:to_pairs_file (478) [100%] 478 of 478 ✔ [05/1f5fd4] process > POREC:merge_pairs (1) [100%] 1 of 1 ✔ [59/bad389] process > POREC:merge_pairs_stats (1) [100%] 1 of 1 ✔ [34/64bd2b] process > POREC:pair_stats_report (1) [100%] 1 of 1 ✔ [95/ab9049] process > POREC:getVersions [100%] 1 of 1 ✔ [c8/dd8f30] process > POREC:getParams [100%] 1 of 1 ✔ [30/f80d1b] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [35/04a9fc] process > POREC:prepare_hic (1) [ 0%] 0 of 1 WARN: Failed to publish file: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/9d/8872caa15ddd7b481debf17e50517c/null.ns.bam; to: /lustre/nobackup/WUR/ABGC/dieps003/9.4/pore-c_3/output/bams/null.ns.bam [copy] -- See log file for details

executor > local (970) [dc/ad9291] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [6f/4ec9fd] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [b1/1f38fc] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d0/65c656] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [11/301a9e] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [19/c08853] process > POREC:merge_coordsorted_bam... [100%] 1 of 1 ✔ [9d/8872ca] process > POREC:merge_namesorted_bams... [100%] 1 of 1 ✔ [aa/53ff40] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [f8/7018cc] process > POREC:to_pairs_file (478) [100%] 478 of 478 ✔ [05/1f5fd4] process > POREC:merge_pairs (1) [100%] 1 of 1 ✔ [59/bad389] process > POREC:merge_pairs_stats (1) [100%] 1 of 1 ✔ [34/64bd2b] process > POREC:pair_stats_report (1) [100%] 1 of 1 ✔ [95/ab9049] process > POREC:getVersions [100%] 1 of 1 ✔ [c8/dd8f30] process > POREC:getParams [100%] 1 of 1 ✔ [30/f80d1b] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [35/04a9fc] process > POREC:prepare_hic (1) [ 0%] 0 of 1

executor > local (970) [dc/ad9291] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [6f/4ec9fd] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [b1/1f38fc] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d0/65c656] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [11/301a9e] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [19/c08853] process > POREC:merge_coordsorted_bam... [100%] 1 of 1 ✔ [9d/8872ca] process > POREC:merge_namesorted_bams... [100%] 1 of 1 ✔ [aa/53ff40] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [f8/7018cc] process > POREC:to_pairs_file (478) [100%] 478 of 478 ✔ [05/1f5fd4] process > POREC:merge_pairs (1) [100%] 1 of 1 ✔ [59/bad389] process > POREC:merge_pairs_stats (1) [100%] 1 of 1 ✔ [34/64bd2b] process > POREC:pair_stats_report (1) [100%] 1 of 1 ✔ [95/ab9049] process > POREC:getVersions [100%] 1 of 1 ✔ [c8/dd8f30] process > POREC:getParams [100%] 1 of 1 ✔ [30/f80d1b] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [35/04a9fc] process > POREC:prepare_hic (1) [ 0%] 0 of 1 ERROR ~ Error executing process > 'POREC:prepare_hic (1)'

Caused by: Process POREC:prepare_hic (1) terminated with an error exit status (1)

Command executed:

cut -f1,2 fasta.fai > sizes.genome pairtools flip input.pairs.gz -c sizes.genome > flipped.pairs.tmp pairtools sort flipped.pairs.tmp > sorted.pairs.tmp pairtools dedup sorted.pairs.tmp > dedup.pairs.tmp java -jar /home/epi2melabs/juicer_tools_1.22.01.jar pre dedup.pairs.tmp "null.hic" sizes.genome rm -rf "*.pairs.tmp"

Command exit status: 1

Command output: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body .............................. Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000

Command error: INFO: Converting SIF file to temporary sandbox... WARNING: underlay of /etc/localtime required more than 50 (78) bind mounts WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body ............................................................................... Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at juicebox.tools.utils.norm.NormalizationCalculations.computeKR(NormalizationCalculations.java:427) at juicebox.tools.utils.norm.NormalizationVectorUpdater.buildKR(NormalizationVectorUpdater.java:209) at juicebox.tools.utils.norm.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:146) at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:151) at juicebox.tools.HiCTools.main(HiCTools.java:94) INFO: Cleaning up image...

Work dir: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/35/04a9fc0ae258c57ae7b4a9db139a87

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

executor > local (970) [dc/ad9291] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [6f/4ec9fd] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [b1/1f38fc] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d0/65c656] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [11/301a9e] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [19/c08853] process > POREC:merge_coordsorted_bam... [100%] 1 of 1 ✔ [9d/8872ca] process > POREC:merge_namesorted_bams... [100%] 1 of 1 ✔ [aa/53ff40] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [f8/7018cc] process > POREC:to_pairs_file (478) [100%] 478 of 478 ✔ [05/1f5fd4] process > POREC:merge_pairs (1) [100%] 1 of 1 ✔ [59/bad389] process > POREC:merge_pairs_stats (1) [100%] 1 of 1 ✔ [34/64bd2b] process > POREC:pair_stats_report (1) [100%] 1 of 1 ✔ [95/ab9049] process > POREC:getVersions [100%] 1 of 1 ✔ [c8/dd8f30] process > POREC:getParams [100%] 1 of 1 ✔ [30/f80d1b] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [35/04a9fc] process > POREC:prepare_hic (1) [100%] 1 of 1, failed: 1 ✘ ERROR ~ Error executing process > 'POREC:prepare_hic (1)'

Caused by: Process POREC:prepare_hic (1) terminated with an error exit status (1)

Command executed:

cut -f1,2 fasta.fai > sizes.genome pairtools flip input.pairs.gz -c sizes.genome > flipped.pairs.tmp pairtools sort flipped.pairs.tmp > sorted.pairs.tmp pairtools dedup sorted.pairs.tmp > dedup.pairs.tmp java -jar /home/epi2melabs/juicer_tools_1.22.01.jar pre dedup.pairs.tmp "null.hic" sizes.genome rm -rf "*.pairs.tmp"

Command exit status: 1

Command output: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body ............................................... Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000

Command error: INFO: Converting SIF file to temporary sandbox... WARNING: underlay of /etc/localtime required more than 50 (78) bind mounts WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body .......................................... Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at juicebox.tools.utils.norm.NormalizationCalculations.computeKR(NormalizationCalculations.java:427) at juicebox.tools.utils.norm.NormalizationVectorUpdater.buildKR(NormalizationVectorUpdater.java:209) at juicebox.tools.utils.norm.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:146) at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:151) at juicebox.tools.HiCTools.main(HiCTools.java:94) INFO: Cleaning up image...

Work dir: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/35/04a9fc0ae258c57ae7b4a9db139a87

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

executor > local (970) [dc/ad9291] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [6f/4ec9fd] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [b1/1f38fc] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d0/65c656] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [11/301a9e] process > POREC:digest_align_annotate... [100%] 478 of 478 ✔ [- ] process > POREC:haplotag_alignments - [19/c08853] process > POREC:merge_coordsorted_bam... [100%] 1 of 1 ✔ [9d/8872ca] process > POREC:merge_namesorted_bams... [100%] 1 of 1 ✔ [aa/53ff40] process > POREC:create_restriction_be... [100%] 1 of 1 ✔ [f8/7018cc] process > POREC:to_pairs_file (478) [100%] 478 of 478 ✔ [05/1f5fd4] process > POREC:merge_pairs (1) [100%] 1 of 1 ✔ [59/bad389] process > POREC:merge_pairs_stats (1) [100%] 1 of 1 ✔ [34/64bd2b] process > POREC:pair_stats_report (1) [100%] 1 of 1 ✔ [95/ab9049] process > POREC:getVersions [100%] 1 of 1 ✔ [c8/dd8f30] process > POREC:getParams [100%] 1 of 1 ✔ [30/f80d1b] process > POREC:makeReport (1) [100%] 1 of 1 ✔ [35/04a9fc] process > POREC:prepare_hic (1) [100%] 1 of 1, failed: 1 ✘ ERROR ~ Error executing process > 'POREC:prepare_hic (1)'

Caused by: Process POREC:prepare_hic (1) terminated with an error exit status (1)

Command executed:

cut -f1,2 fasta.fai > sizes.genome pairtools flip input.pairs.gz -c sizes.genome > flipped.pairs.tmp pairtools sort flipped.pairs.tmp > sorted.pairs.tmp pairtools dedup sorted.pairs.tmp > dedup.pairs.tmp java -jar /home/epi2melabs/juicer_tools_1.22.01.jar pre dedup.pairs.tmp "null.hic" sizes.genome rm -rf "*.pairs.tmp"

Command exit status: 1

Command output: WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body ......................................................... Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000

Command error: INFO: Converting SIF file to temporary sandbox... WARNING: underlay of /etc/localtime required more than 50 (78) bind mounts WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. WARN [2023-10-15T04:01:24,169] [Globals.java:138] [main] Development mode is enabled Using 1 CPU thread(s) Not including fragment map Start preprocess Writing header Writing body .................................... Writing footer

Finished preprocess

Calculating norms for zoom BP_2500000 Calculating norms for zoom BP_1000000 Calculating norms for zoom BP_500000 Calculating norms for zoom BP_250000 Calculating norms for zoom BP_100000 Calculating norms for zoom BP_50000 Calculating norms for zoom BP_25000 Calculating norms for zoom BP_10000Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at juicebox.tools.utils.norm.NormalizationCalculations.computeKR(NormalizationCalculations.java:427) at juicebox.tools.utils.norm.NormalizationVectorUpdater.buildKR(NormalizationVectorUpdater.java:209) at juicebox.tools.utils.norm.NormalizationVectorUpdater.updateHicFile(NormalizationVectorUpdater.java:146) at juicebox.tools.clt.old.PreProcessing.run(PreProcessing.java:151) at juicebox.tools.HiCTools.main(HiCTools.java:94) INFO: Cleaning up image...

Work dir: /lustre/nobackup/WUR/ABGC/dieps003/9.4/work/35/04a9fc0ae258c57ae7b4a9db139a87

Tip: view the complete command output by changing to the process work dir and entering the command cat .command.out

-- Check '.nextflow.log' file for details

sarahjeeeze commented 1 year ago

Hi, looks like its running out of memory at that step, would you be able to share what the pore-c_config.config you are using looks like?

SwenDiepstraten commented 1 year ago

yes ofcourse,

executor { $local { cpus = 32 memory = "350000" } }

sarahjeeeze commented 1 year ago

for memory i would try something like "16 GB"

SwenDiepstraten commented 1 year ago

I am trying this but it takes a long time to run, will update when it is finished!

SwenDiepstraten commented 1 year ago

Unfortunately I am still encountering this error, even with the updated memory

sarahjeeeze commented 1 year ago

Ok thanks for reporting, I will investigate further and get back to you.

sarahjeeeze commented 1 year ago

Hi, thanks again for reporting this. I did some research and think I need to expose a few of the dedup parameters which should allow you to reduce the memory requirement for this step. Sorry I think perhaps you have a larger dataset than we have so far tested it with. I will try to get this done as soon as possible for you to test.

Oksanak22 commented 1 year ago

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [9f/4d43d0] process > POREC:digest_align_annotate... [ 95%] 951 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [1f/e2877c] process > POREC:digest_align_annotate... [ 95%] 952 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [7f/17f85e] process > POREC:digest_align_annotate... [ 95%] 954 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [f0/875019] process > POREC:digest_align_annotate... [ 95%] 955 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [27/7f21a8] process > POREC:digest_align_annotate... [ 96%] 956 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [02/a98bec] process > POREC:digest_align_annotate... [ 96%] 957 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [31/97c6f6] process > POREC:digest_align_annotate... [ 96%] 961 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [89/8bbc58] process > POREC:digest_align_annotate... [ 96%] 962 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [cc/05f874] process > POREC:digest_align_annotate... [ 96%] 963 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [49/006c0e] process > POREC:digest_align_annotate... [ 96%] 965 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [a1/65019a] process > POREC:digest_align_annotate... [ 97%] 968 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [35/a6aa53] process > POREC:digest_align_annotate... [ 97%] 970 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [2c/6d84e1] process > POREC:digest_align_annotate... [ 97%] 971 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [1b/0e4b1f] process > POREC:digest_align_annotate... [ 97%] 972 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [b1/14fd64] process > POREC:digest_align_annotate... [ 97%] 974 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [fd/e6039f] process > POREC:digest_align_annotate... [ 98%] 977 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/4e09be] process > POREC:digest_align_annotate... [ 98%] 980 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [37/8be295] process > POREC:digest_align_annotate... [ 98%] 982 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [84/6cecac] process > POREC:digest_align_annotate... [ 98%] 985 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [d7/2e6e57] process > POREC:digest_align_annotate... [ 99%] 989 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [af/ec6bae] process > POREC:digest_align_annotate... [ 99%] 992 of 995 [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 66%] 2 of 3, failed: 2... [- ] process > POREC:prepare_hic -

executor > slurm (1004) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [7a/9e0cbb] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1005) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1005) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [- ] process > POREC:merge_namesorted_bams [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1006) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams - [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... - [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1006) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [- ] process > POREC:merge_parquets_to_dat... [ 0%] 0 of 1 [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1007) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bams [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [ 0%] 0 of 1 [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1008) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [a8/e5d7e1] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [ 0%] 0 of 1 [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - [7a/9e0cbb] NOTE: Process POREC:makeReport (1) terminated with an error exit status (1) -- Execution is retried (3)

executor > slurm (1008) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [a8/e5d7e1] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [ 0%] 0 of 1 [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic -

executor > slurm (1008) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [a8/e5d7e1] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [100%] 1 of 1 ✔ [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic -

executor > slurm (1008) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [a8/e5d7e1] process > POREC:merge_coordsorted_bam... [ 0%] 0 of 1 [df/9d49ff] process > POREC:merge_namesorted_bams... [ 0%] 0 of 1 [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [100%] 1 of 1 ✔ [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [ 75%] 3 of 4, failed: 3... [- ] process > POREC:prepare_hic - ERROR ~ Error executing process > 'POREC:makeReport (1)'

Caused by: Process POREC:makeReport (1) terminated with an error exit status (1)

Command executed:

echo '[ { "barcode": null, "type": "test_sample", "alias": "N141N144" } ]' > metadata.json workflow-glue report wf-pore-c-report.html --metadata metadata.json --stats per_read_stats.tsv --versions versions --params params.json

Command exit status: 1

Command output: (empty)

Command error: INFO: underlay of /etc/localtime required more than 50 (78) bind mounts [21:35:45 - workflow_glue] Starting entrypoint. Traceback (most recent call last): File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow-glue", line 7, in cli() File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow_glue/init.py", line 72, in cli args.func(args) File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow_glue/report.py", line 31, in main fastcat.SeqSummary(args.stats) File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 47, in init df_all = load_stats(seq_summary, format='fastcat') File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 375, in load_stats df['start_time'] = df.start_time.dt.tz_localize(None) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/generic.py", line 5902, in getattr return object.getattribute(self, name) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/accessor.py", line 182, in get accessor_obj = self._accessor(obj) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/indexes/accessors.py", line 512, in new raise AttributeError("Can only use .dt accessor with datetimelike values") AttributeError: Can only use .dt accessor with datetimelike values

Work dir: /lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/work/61/aa1ae333eacc39f395445ef22bd5a7

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

-- Check '.nextflow.log' file for details WARN: Killing running tasks (2)

executor > slurm (1008) [f9/520707] process > POREC:fastcat (1) [100%] 1 of 1 ✔ [e1/d3be3c] process > POREC:index_ubam (1) [100%] 1 of 1 ✔ [93/1d2dea] process > POREC:preparegenome:index... [100%] 1 of 1, failed: 1 ✔ [28/b43779] process > POREC:preparegenome:index... [100%] 1 of 1 ✔ [dd/c90018] process > POREC:digest_align_annotate... [100%] 995 of 995 ✔ [- ] process > POREC:haplotag_alignments - [- ] process > POREC:merge_coordsorted_bam... - [- ] process > POREC:merge_namesorted_bams... - [- ] process > POREC:create_restriction_bed - [- ] process > POREC:to_pairs_file - [- ] process > POREC:pairsToCooler - [- ] process > POREC:merge_mcools - [- ] process > POREC:merge_pairs - [- ] process > POREC:merge_pairs_stats - [- ] process > POREC:pair_stats_report - [72/3259fd] process > POREC:merge_parquets_to_dat... [100%] 1 of 1 ✔ [2e/f2af9c] process > POREC:getVersions [100%] 1 of 1 ✔ [0a/73e3f4] process > POREC:getParams [100%] 1 of 1 ✔ [61/aa1ae3] process > POREC:makeReport (1) [100%] 4 of 4, failed: 4... [- ] process > POREC:prepare_hic - ERROR ~ Error executing process > 'POREC:makeReport (1)'

Caused by: Process POREC:makeReport (1) terminated with an error exit status (1)

Command executed:

echo '[ { "barcode": null, "type": "test_sample", "alias": "N141N144" } ]' > metadata.json workflow-glue report wf-pore-c-report.html --metadata metadata.json --stats per_read_stats.tsv --versions versions --params params.json

Command exit status: 1

Command output: (empty)

Command error: INFO: underlay of /etc/localtime required more than 50 (78) bind mounts [21:35:45 - workflow_glue] Starting entrypoint. Traceback (most recent call last): File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow-glue", line 7, in cli() File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow_glue/init.py", line 72, in cli args.func(args) File "/lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/bin/workflow_glue/report.py", line 31, in main fastcat.SeqSummary(args.stats) File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 47, in init df_all = load_stats(seq_summary, format='fastcat') File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 375, in load_stats df['start_time'] = df.start_time.dt.tz_localize(None) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/generic.py", line 5902, in getattr return object.getattribute(self, name) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/accessor.py", line 182, in get accessor_obj = self._accessor(obj) File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/indexes/accessors.py", line 512, in new raise AttributeError("Can only use .dt accessor with datetimelike values") AttributeError: Can only use .dt accessor with datetimelike values

Work dir: /lustre04/scratch/oksanak/wf-pore-c-0.1.0/wf-pore-c/work/61/aa1ae333eacc39f395445ef22bd5a7

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

-- Check '.nextflow.log' file for details

Oksanak22 commented 1 year ago

Unfortunately, the previous run went fine, but this one is like this. I just changed the reference genome. Even from bigger to smaller.

Thank you.

sarahjeeeze commented 1 year ago

Hi, sorry, as mentioned I think we need to add a fix to the workflow for this. Do you know how large your input fastq or bam files are?

Oksanak22 commented 1 year ago

Hello. My .fastq is 52G. But I have analyzed this .fastq before successfully. In this analysis, I changed a reference genom.fastq file (not the whole, but just a subset for part B of chromosomes). And it give me this error or

[- ] process > POREC:pair_stats_report -

[- ] process > POREC:merge_parquets_to_dat... -

[82/d39e1a] process > POREC:getVersions [100%] 1 of 1 ✔

[ec/344f1c] process > POREC:getParams [100%] 1 of 1 ✔

[- ] process > POREC:makeReport -

[- ] process > POREC:prepare_hic -

[7e/bf2881] NOTE: Missing output file(s) seqs.fastq.gz expected by process POREC:fastcat (1) -- Error is ignored

executor > slurm (5)

[7e/bf2881] process > POREC:fastcat (1) [100%] 1 of 1, failed: 1 ✔

[- ] process > POREC:index_ubam -

[81/f0e5dd] process > POREC:preparegenome:index... [100%] 1 of 1 ✔

[e8/c141e7] process > POREC:preparegenome:index... [100%] 1 of 1 ✔

[- ] process > POREC:digest_align_annotate -

[- ] process > POREC:haplotag_alignments -

[- ] process > POREC:merge_coordsorted_bams -

[- ] process > POREC:merge_namesorted_bams -

[- ] process > POREC:create_restriction_bed -

[- ] process > POREC:to_pairs_file -

[- ] process > POREC:pairsToCooler -

[- ] process > POREC:merge_mcools -

[- ] process > POREC:merge_pairs -

[- ] process > POREC:merge_pairs_stats -

[- ] process > POREC:pair_stats_report -

[- ] process > POREC:merge_parquets_to_dat... -

[82/d39e1a] process > POREC:getVersions [100%] 1 of 1 ✔

[ec/344f1c] process > POREC:getParams [100%] 1 of 1 ✔

[- ] process > POREC:makeReport -

[- ] process > POREC:prepare_hic -

Completed at: 27-Oct-2023 22:23:28

Duration : 5h 13m 17s

CPU hours : 20.4 (100% failed)

Succeeded : 4

Ignored : 1

Failed : 1

Oksanak22 commented 1 year ago

yes, for sure Here how it looks like --ref /home/nak/Genome/B_fork.fasta

sarahjeeeze commented 1 year ago

Hi, sorry, does it break at the same place every time?

Oksanak22 commented 1 year ago

Mostly at step "making report". But in the output folder I have just "execution" folder:

drwxr-x---. 3 ak ak 25600 Oct 27 17:10 . drwxr-x---. 24 ak ak 33792 Oct 30 10:21 .. drwxr-x---. 2 ak ak 25600 Oct 27 22:23 execution

sarahjeeeze commented 1 year ago

Hi, just to check it is not something wrong with an older version of one of the programmes the report process uses could you retry it using the prerelease revision by adding -r prerelease

Oksanak22 commented 1 year ago

Sorry, I didn't catch completely where I have add "-r prerelease"? In my script for the job submissions or where?

sarahjeeeze commented 1 year ago

Oh sorry, add it to the command line ... so nextflow run... -r prerelease. Thanks!

Oksanak22 commented 1 year ago

N E X T F L O W ~ version 23.04.3 Revision option cannot be used when running a local script

sarahjeeeze commented 1 year ago

Oh sorry, it only works with the nextflow run epi2me-labs/wf-pore-c... command.

Oksanak22 commented 1 year ago

Hello. But I am running my analysis on a cluster.

cjw85 commented 1 year ago

Hi @Oksanak22,

Could you please open a new issue rather than co-opting this unrelated issue from another user.

Oksanak22 commented 1 year ago

Yes, @cjw85 I made it https://github.com/epi2me-labs/wf-pore-c/issues/37

cjw85 commented 1 year ago

@SwenDiepstraten,

Have you managed to resolve your issue?

sarahjeeeze commented 11 months ago

Hi, we have since added a fix for this merge_pairs issue which is available in the latest version.

sarahjeeeze commented 9 months ago

Closing through lack of response, and issue should be resolved. Let us know if you have any other issues.