pipe error with bwa-mem2.avx512bw (2.2.1)

[pashapu43]

Hi,

I am trying to align paired-end HI-C data with bwa-mem2 but it seems there is an issue with header. Here is the command i used: bwa-mem2.avx512bw mem -5SP -T0 -t50 -R "@RG\tID:E003868R:379:GW2111194210th:3\tSM:Tim_omnic\tPL:ILLUMINA\tLB:Lib1" \ pinhic_tim DTG-OmniC-72_90_R1.fastq.gz DTG-OmniC-72_90_R2.fastq.gz| \ pairtools parse --min-mapq 30 --walks-policy 5unique --max-inter-align-gap 30 --nproc-in 20 --nproc-out 20 --chroms-path pinhic.genome | \ pairtools sort --tmpdir=./pinhic_tmp --nproc 20|pairtools dedup --nproc-in 20 --nproc-out 20 --mark-dups --output-stats stats.txt| \ pairtools split --nproc-in 20 --nproc-out 20 --output-pairs mapped.pairs --output-sam -|samtools view -bS -@16 | samtools sort -@16 -o pinhic_aligned.bam;samtools index pinhic_aligned.bam.

The same command works without any issue with bwa mem (0.7.17-r1188).

Here is the SLRUM output:

Executing in AVX512 mode!!

• SA compression enabled with xfactor: 8

• Ref file: pinhic_tim

• Entering FMI_search

• Index file found. Loading index from pinhic_tim.bwt.2bit.64

• Reference seq len for bi-index = 8647430785

• sentinel-index: 3644441659

• Count: 0, 1 1, 2364673030 2, 4323715393 3, 6282757756 4, 8647430785

• Reading other elements of the index from files pinhic_tim

• Index prefix: pinhic_tim Traceback (most recent call last): File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/bin/pairtools", line 11, in sys.exit(cli()) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/init.py", line 183, in wrapper return func(*args, *kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/parse.py", line 165, in parse parse_py( File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/parse.py", line 176, in parse_py input_sam = AlignmentFilePairtoolized( File "pysam/libcalignmentfile.pyx", line 748, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 953, in pysam.libcalignmentfile.AlignmentFile._open ValueError: file does not contain alignment data WARNING:pairtools:Headerless input, please, add the header by pairtools header generate or pairtools header transfer Traceback (most recent call last): File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/bin/pairtools", line 11, in sys.exit(cli()) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 783, in invoke return callback(*args, **kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/init__.py", line 183, in wrapper return func(*args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/sort.py", line 70, in sort sort_py(pairs_path, output, nproc, tmpdir, memory, compress_program, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/sort.py", line 89, in sort_py header = headerops.append_new_pg(header, ID=UTIL_NAME, PN=UTIL_NAME) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/lib/headerops.py", line 346, in append_new_pg raise Exception("Input file is not valid .pairs, has no header or is empty.") Exception: Input file is not valid .pairs, has no header or is empty. WARNING:pairtools:Headerless input, please, add the header by pairtools header generate or pairtools header transfer WARNING:pairtools:Pairs file appears not to be sorted, dedup might produce wrong results. Traceback (most recent call last): File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/bin/pairtools", line 11, in sys.exit(cli()) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/init.py", line 183, in wrapper return func(*args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/dedup.py", line 326, in dedup dedup_py( File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/dedup.py", line 488, in dedup_py header = headerops.append_new_pg(header, ID=UTIL_NAME, PN=UTIL_NAME) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/lib/headerops.py", line 346, in append_new_pg raise Exception("Input file is not valid .pairs, has no header or is empty.") Exception: Input file is not valid .pairs, has no header or is empty. WARNING:pairtools:Headerless input, please, add the header by pairtools header generate or pairtools header transfer Traceback (most recent call last): File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/bin/pairtools", line 11, in sys.exit(cli()) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1157, in call return self.main(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, ctx.params) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/click/core.py", line 783, in invoke return __callback(args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/init.py", line 183, in wrapper return func(*args, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/split.py", line 44, in split split_py(pairsam_path, output_pairs, output_sam, kwargs) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/cli/split.py", line 80, in split_py header = headerops.append_new_pg(header, ID=UTIL_NAME, PN=UTIL_NAME) File "/mnt/users/akhpasha/.conda/envs/mbase/envs/aligners/lib/python3.10/site-packages/pairtools/lib/headerops.py", line 346, in append_new_pg raise Exception("Input file is not valid .pairs, has no header or is empty.") Exception: Input file is not valid .pairs, has no header or is empty. [main_samview] fail to read the header from "-". [W::hts_set_opt] Cannot change block size for this format samtools sort: failed to read header from "-" [E::hts_open_format] Failed to open file "pinhic_aligned.bam" : No such file or directory samtools index: failed to open "pinhic_aligned.bam": No such file or directory

[golobor]

did you check if the bwa mem2 command runs well and generates valid .sam/.bam files? For debugging, I recommend running alignment and parsing as separate commands.