binning.sh: line 242: syntax error near unexpected token

jjjscuedu commented 5 years ago

Dear all,

When I try to run metaWRAP for my dataset,

metawrap binning -o INITIAL_BINNING -t 40 -a /gpfs02/bioinfo/metagenome_data/Assembly_Merge/gp4.final.contigs.fa --metabat2 --maxbin2 --concoct /gpfs02/bioinfo/metagenome_data/Reads_Filted/gp4/gp4*.fastq

there are some errors like this:

bwa_index] Pack forward-only FASTA... 17.02 sec
[bwa_index] Construct SA from BWT and Occ... 1185.70 sec
[main] Version: 0.7.15-r1140
[main] CMD: bwa index INITIAL_BINNING/work_files/assembly.fa
[main] Real time: 4782.748 sec; CPU: 4756.714 sec
/gpfs02/home/jingjing/miniconda2/envs/metawrap-env/bin/metawrap-modules/binning.sh: line 242: syntax error near unexpected token `|'
/gpfs02/home/jingjing/miniconda2/envs/metawrap-env/bin/metawrap-modules/binning.sh: line 242: `             | samtools view -@ $threads -bS - \'

i am not sure for the reason and chek line 242 for this binning.sh file:

bwa mem -t $threads ${out}/work_files/assembly.fa $reads_1 $reads_2 \
                #| tee >( awk '{ if ($9 > 0) { N+=1; S+=$9; S2+=$9*$9 }} END { M=S/N; print ""N"\t "M"\t "sqrt ((S2-M*M*N)/(N-1))}'\
                #>> ${out}/insert_sizes.txt ) \
                | samtools view -@ $threads -bS - \
                | samtools sort -T ${out}/work_files/tmp-samtools -@ $threads -O bam \
                -o ${out}/work_files/${sample}.bam -

Can anyone give me some suggestions?

Thanks a lot!

Jingjing

ursky commented 5 years ago

I am not sure why this syntax is giving you issues. Perhaps a different bash version? Could you try removing the commented out lines and see if that works? Like this:

bwa mem -t $threads ${out}/work_files/assembly.fa $reads_1 $reads_2 \
| samtools view -@ $threads -bS - \
| samtools sort -T ${out}/work_files/tmp-samtools -@ $threads -O bam \
-o ${out}/work_files/${sample}.bam -

If that doesn't work, then mayby put them on one line:

bwa mem -t $threads ${out}/work_files/assembly.fa $reads_1 $reads_2 | samtools view -@ $threads -bS - | samtools sort -T ${out}/work_files/tmp-samtools -@ $threads -O bam -o ${out}/work_files/${sample}.bam -

chloelulu commented 5 years ago

Hi, @ursky ,

I met exactly the same problem as @jjjscuedu .(Shown in uploaded file) My bash version is echo "${BASH_VERSION}" 4.2.46(2)-release. And the bug has been resolved by removing the commented out lines. Thanks so much! error.txt

ursky commented 5 years ago

Thanks for the info! I had a few users contact me about this. The issue seems to be system-dependent - I cannot replicate it. This confuses me, because I have the same bash version... Nonetheless, its an easy fix, which I incorporated into metawrap v1.1.1, which is now out.

astrophys commented 5 years ago

When will this update get pushed to conda? It seems to still be on version 1.0. Updating via conda install metawrap-mg=1.1.1 gets stuck indefinitely in Solving environment (I stopped after 40minutes in this state).

ursky commented 5 years ago

I would leave it running longer than that. Conda can take a long time depending on the system and software. MetaWRAP has so many dependencies that Conda struggles to find a configuration that satisfies all the software versions. Hence the long Solving environment stage. The versions 1.0 that you are seeing is the one on Bioconda - which does not include the latest updates.

bxlab / metaWRAP

binning.sh: line 242: syntax error near unexpected token #91