search

marbl / MetaCompass

MetaCompass: Reference-guided Assembly of Metagenomes
https://github.com/marbl/MetaCompass/wiki
Other
38 stars 11 forks source link

one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode #17

Open MarcelaHer opened 4 years ago

MarcelaHer commented 4 years ago

Hi, what does this mean? is it that the quality of my reads are bad or that I don't have any MAG related to the ref genome I am searching for? Thanks!

Job 2: ---assembly stats reference-guided contigs

python3 /gpfs/software/ada/build/MetaCompass-2.0-beta/bin/assembly_stats.py metacompass_heavyadmintest/metacompass_output/metacompass.final.ctg.fa 1 > metacompass_heavyadmintest/metacompass_output/metacompass_assembly_stats.tsv && touch metacompass_heavyadmintest/metacompass_output/.run2.ok

Error in rule stats_all: jobid: 2 output: metacompass_heavyadmintest/metacompass_output/metacompass_assembly_stats.tsv shell: python3 /gpfs/software/ada/build/MetaCompass-2.0-beta/bin/assembly_stats.py metacompass_heavyadmintest/metacompass_output/metacompass.final.ctg.fa 1 > metacompass_heavyadmintest/metacompass_output/metacompass_assembly_stats.tsv && touch metacompass_heavyadmintest/metacompass_output/.run2.ok (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job stats_all since they might be corrupted: metacompass_heavyadmintest/metacompass_output/metacompass_assembly_stats.tsv

vcepeda commented 4 years ago

Hi, can you check if the assembly/contigs.fasta, assembly/megahit/final.contig.fa, or metacompass_output/metacompass.final.ctg.fa files are empy? Also, can you share the last log file in the log folder?

MarcelaHer commented 4 years ago

Hi,

the assembly/contigs.fasta is 3,4 MB the assembly/megahit/final.contig.fa 721,5 MB the metacompass_output/metacompass.final.ctg.fa is 724,4 MB

I can't share the final log (pilon.log) as it is 34 Mb.

I can share the megahit.log megahit.log

Cheers

MarcelaHer commented 4 years ago

Hi, I run it again and got the same error. Please, I would be grateful if you can help me with this. Any idea why I am having this issue? Cheers!

vcepeda commented 4 years ago

Can you post the command you ran? and also did you run the tutorial successfully?

MarcelaHer commented 4 years ago

For my run I did: go_metacompass2.py -r Luteitalea_pratensis.fasta -1 heavyclean_250_NewHeadersR_1.fastq -2 heavyclean_250_NewHeadersR_2.fastq -o metacompass_heavytest2 -m 1 -y 80

Now, for the tutorial: also didn't run, I did: go_metacompass2.py -r Candidatus_Carsonella_ruddii_HT_Thao2000.fasta -1 thao2000.1.fq -2 thao2000.2.fq -l 150 -o example1_output -t 4 -y 8 and I got: Removing output files of failed job bowtie2_map since they might be corrupted the tutorial log file: 2020-09-03T135809.115304.snakemake.log

vcepeda commented 4 years ago

Please check if your bowtie2 version is (>=) 2.2.9 and share the file example1_output/logs/bowtie2map.log

MarcelaHer commented 4 years ago

Hi, thanks for your fast reply. my version of Bowtie2 is 2.3.5.1. and the folder "example1_output/logs" is empty

vcepeda commented 4 years ago

Bowtie2 is not working, can you run ls -lrt example1_output/assembly and share the output?

MarcelaHer commented 4 years ago

Hi, the "example1_output/assembly" is also zero I run: ls -lrt example1_output/assembly and total is zero should I re-install Bowtie2?

vcepeda commented 4 years ago

Do you get any output? can you ls all the folders from example1_output?

You should get some output when running "bowtie2-build -o 3 --threads 4 -q Candidatus_Carsonella_ruddii_HT_Thao2000.fasta example1_output/assembly/mc.index 1>> example1_output/assembly/mc.index 2>&1". Please try to reinstall bowtie2. Older versions of Bowtie2 don't have the --threads option

MarcelaHer commented 4 years ago

Hi, I got this output after running your command. It says reference seems not to be on a fasta, but I am double sure it is in fasta format.

mc.index.txt

vcepeda commented 4 years ago

Is seems like there is an issue with Bowtie2, not the metacompass installation. Can you run "bowtie2-build -o 3 --threads 4 -q Candidatus_Carsonella_ruddii_HT_Thao2000.fasta mc.index"

MarcelaHer commented 4 years ago

Hi, our HPC expert did a run and they think is a problem with the print issues with python3.

Also, they asked me why sometimes it is calling python3 and python2 in different python scripts? grep python *.py assembly.py:#!/usr/bin/env python3 assembly_stats.py: Usage: python assembly_stats.py assembly_validation.py:#!/usr/bin/env python best-hits.py:#!/usr/bin/env python3 best_strata.py:#!/usr/bin/env python3 go_metacompass.py:#!/usr/bin/env python3 go_metacompass2.py:#!/usr/bin/env python select_references.py:#!/usr/bin/env python

the script of the run: go_metacompass2.py -r Candidatus_Carsonella_ruddii_HT_Thao2000.fasta -1 thao2000.1.fq -2 thao2000.2.fq -l 150 -o example8amjad_output -t 4 -y 8

The error showed this Error in rule stats_all: jobid: 2 output: example7amjad_output/metacompass_output/metacompass_assembly_stats.tsv shell: python3 /gpfs/software/ada/build/metacompass/MetaCompass-2.0-beta/bin/assembly_stats.py example7amjad_output/metacompass_output/metacompass.final.ctg.fa 1 > example7amjad_output/metacompass_output/metacompass_assembly_stats.tsv && touch example7amjad_output/metacompass_output/.run2.ok (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job stats_all since they might be corrupted: example7amjad_output/metacompass_output/metacompass_assembly_stats.tsv [Wed Sep 16 11:08:37 2020] Finished job 1. 11 of 14 steps (79%) done File "/gpfs/software/ada/build/metacompass/MetaCompass-2.0-beta/bin/assembly_stats.py", line 127 print 'Filename: ' + str(contigsMultifasta)
^ SyntaxError: invalid syntax [Wed Sep 16 11:08:37 2020] Finished job 3. 12 of 14 steps (86%) done Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /gpfs/software/ada/build/metacompass/MetaCompass-2.0-beta/tutorial/.snakemake/log/2020-09-16T110744.049416.snakemake.log

vcepeda commented 4 years ago

All scripts were written for python3. It seems like you have python 2 and 3 installed, and your pythonpath is pointed at python 2. Please ask your HPC expert how to fix that problem. A quick fix can be running the command alias python=python3

MarcelaHer commented 4 years ago

Hi, Our HPC expert checked it and Python3 is in python path when metacompass module is loaded MetaCompass/2.0-beta echo $PYTHONPATH /gpfs/software/ada/python/anaconda/2019.10/3.7

so I am not sure why we are having the problem...

vcepeda commented 4 years ago

I was having a similar issue with an older version of my university system before and running alias python=python3 fixed it, can you try that to see if the tutorial data can run successfully?

vcepeda commented 4 years ago

I have a new version of the code that haven't pushed to Github that will probably solve your issues.

MarcelaHer commented 4 years ago

great, thank you. When do you think we can have access to it? Cheers.

MarcelaHer commented 4 years ago

Hi, when do you think you can upload the new version? Cheers!