metagenome-atlas / atlas

ATLAS - Three commands to start analyzing your metagenome data
https://metagenome-atlas.github.io/
BSD 3-Clause "New" or "Revised" License
377 stars 99 forks source link

Metatranscriptome analysis issues #170

Closed MCamp91 closed 5 years ago

MCamp91 commented 5 years ago

Hello,

I’m trying to run RNAseq data through atlas v2.0.1 for metatranscriptome analysis. I’ve tested both assemblers and have ran into different issues (see logs below). I’ve been advised that the binning step may not work for this type of data and that the binning step should be skipped, I think this is the issue with the metahit run. Is there a way to skip the binning step and move directly to the annotation step?

Is there any parameters that could be changed in the config to successfully run this data for metatranscriptomics?

Cheers,

Metahit 109 of 209 steps (52%) done [Tue Feb 12 17:07:54 2019] rule maxbin: input: MM1-SB/MM1-SB_contigs.fasta, MM1-SB/binning/coverage/MM1-SB_coverage.txt output: MM1-SB/binning/maxbin/intermediate_files log: MM1-SB/logs/binning/maxbin.log jobid: 167 wildcards: sample=MM1-SB threads: 16 mkdir MM1-SB/binning/maxbin/intermediate_files 2> MM1-SB/logs/binning/maxbin.log run_MaxBin.pl -contig MM1-SB/MM1-SB_contigs.fasta -abund MM1-SB/binning/coverage/MM1-SB_coverage.txt -out MM1-SB/binning/maxbin/intermediate_files/MM1-SB -min_contig_length 1000 -thread 16 -prob_threshold 0.9 -max_iteration 50 >> MM1-SB/logs/binning/maxbin.log

    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.summary MM1-SB/binning/maxbin/intermediate_files/.. 2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.marker MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.marker_of_each_bin.tar.gz MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.log MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log

Activating conda environment: /databases/conda_envs/189bcd05 [Tue Feb 12 17:08:00 2019] Error in rule maxbin: jobid: 167 output: MM1-SB/binning/maxbin/intermediate_files log: MM1-SB/logs/binning/maxbin.log conda-env: /databases/conda_envs/189bcd05

RuleException: CalledProcessError in line 212 of /opt/conda/lib/python3.6/site-packages/atlas/rules/binning.snakefile: Command 'source activate /databases/conda_envs/189bcd05; set -euo pipefail; mkdir MM1-SB/binning/maxbin/intermediate_files 2> MM1-SB/logs/binning/maxbin.log run_MaxBin.pl -contig MM1-SB/MM1-SB_contigs.fasta -abund MM1-SB/binning/coverage/MM1-SB_coverage.txt -out MM1-SB/binning/maxbin/intermediate_files/MM1-SB -min_contig_length 1000 -thread 16 -prob_threshold 0.9 -max_iteration 50 >> MM1-SB/logs/binning/maxbin.log

    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.summary MM1-SB/binning/maxbin/intermediate_files/.. 2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.marker MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.marker_of_each_bin.tar.gz MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log
    mv MM1-SB/binning/maxbin/intermediate_files/MM1-SB.log MM1-SB/binning/maxbin/intermediate_files/..  2>> MM1-SB/logs/binning/maxbin.log ' returned non-zero exit status 255.

File "/opt/conda/lib/python3.6/site-packages/atlas/rules/binning.snakefile", line 212, in __rule_maxbin File "/opt/conda/lib/python3.6/concurrent/futures/thread.py", line 56, in run Removing output files of failed job maxbin since they might be corrupted: MM1-SB/binning/maxbin/intermediate_files

Spades 36 of 209 steps (17%) done Waiting at most 5 seconds for missing files. MissingOutputException in line 277 of /opt/conda/lib/python3.6/site-packages/atlas/rules/assemble.snakefile: Missing files after 5 seconds: MM1-SB/assembly/contigs.fasta MM1-SB/assembly/scaffolds.fasta This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait. Shutting down, this might take some time.

SilasK commented 5 years ago

Try to use spades and to configure atlas as described in #143

SilasK commented 5 years ago

Skip binning is a bit more complicated because I tried to have a working pipeline for metagenomes first.

Change the source for the gene catalog in the config file.

########################
# Gene catalog
#######################
genecatalog:
  source: contigs  # which predicted proteins should be used for the gene catalog
  clustermethod: linclust # cd-hit-est or cluster or linclust see mmseqs for more details
  minlength_nt: 100
  minid: 0.95

Then run atlas run genecatalog.

MCamp91 commented 5 years ago

Metatranscriptome_metaSpades_run.txt Metagenome_run.txt Metatranscriptomics_rnaSpades_run.txt