KwanLab / Autometa

Autometa: Automated Extraction of Genomes from Shotgun Metagenomes
https://autometa.readthedocs.io
Other
40 stars 15 forks source link

Pipeline fails if number of splits is greater than number of input contigs #205

Open chasemc opened 2 years ago

chasemc commented 2 years ago

e.g. Input parameter is set to --num_splits 12 but only 8 contigs are in the input

Error executing process > 'AUTOMETA:PRODIGAL (Annotating mock_metagenome)'

Caused by:
  Process `AUTOMETA:PRODIGAL (Annotating mock_metagenome)` terminated with an error exit status (18)

Command executed:

  prodigal -i mock_metagenome.filtered.part_013.fasta \
      -p meta -m \
      -f gbk \
      -d "mock_metagenome.fna" \
      -o "mock_metagenome.gbk" \
      -a "mock_metagenome.faa" \
      -s "mock_metagenome_all.txt" 
executor >  local (31)
[55/605a6e] process > AUTOMETA:CREATE_MOCK:GET_ASSEMBLY_SUMMARY                                  [100%] 1 of 1 ✔
[db/be87b1] process > AUTOMETA:CREATE_MOCK:GET_FTP_DIRS (1)                                      [100%] 1 of 1 ✔
[3c/57f78f] process > AUTOMETA:CREATE_MOCK:DOWNLOAD_MOCK_DATA (1)                                [100%] 1 of 1 ✔
[c1/698dae] process > AUTOMETA:CREATE_MOCK:WRITE_FILE_APPEND_COV (1)                             [100%] 1 of 1 ✔
[4b/f4fb85] process > AUTOMETA:SEQKIT_FILTER (Removing contigs < 3000 bp, from mock_metagenome)  [100%] 1 of 1 ✔
[18/9b1d73] process > AUTOMETA:SEQKIT_SPLIT (Splitting mock_metagenome for parallel processing)  [100%] 1 of 1 ✔
[30/1eb8ec] process > AUTOMETA:SPADES_KMER_COVERAGE (Calculating k-mer coverage for mock_meta... [ 85%] 11 of 13
[18/0e8db9] process > AUTOMETA:PRODIGAL (Annotating mock_metagenome)                             [100%] 13 of 13, failed:...
[-        ] process > AUTOMETA:MERGE_PRODIGAL                                                    -
[c0/af0ab7] process > AUTOMETA:ANALYZE_KMERS (Counting kmers for mock_metagenome)                [100%] 1 of 1 ✔
[-        ] process > AUTOMETA:MARKERS                                                           -
[-        ] process > AUTOMETA:MERGE_SPADES_COVERAGE_TSV                                         -
[-        ] process > AUTOMETA:MERGE_HMMSEARCH                                                   -
[-        ] process > AUTOMETA:BINNING:BIN_CONTIGS                                               -
[-        ] process > AUTOMETA:BINNING:BINNING_SUMMARY                                           -
Execution cancelled -- Finishing pending tasks before exit
Error executing process > 'AUTOMETA:PRODIGAL (Annotating mock_metagenome)'

Caused by:
  Process `AUTOMETA:PRODIGAL (Annotating mock_metagenome)` terminated with an error exit status (18)

Command executed:

  prodigal -i mock_metagenome.filtered.part_013.fasta \
      -p meta -m \
      -f gbk \
      -d "mock_metagenome.fna" \
      -o "mock_metagenome.gbk" \
      -a "mock_metagenome.faa" \
      -s "mock_metagenome_all.txt" 

  echo $(prodigal -v 2>&1) | sed -n 's/Prodigal V\(.*\):.*/\1/p' > prodigal.version.txt

Command exit status:
  18

Command output:
  (empty)

Command error:
  -------------------------------------
  PRODIGAL v2.6.3 [February, 2016]         
  Univ of Tenn / Oak Ridge National Lab
  Doug Hyatt, Loren Hauser, et al.     
  -------------------------------------
  Request:  Metagenomic, Phase:  Training
  Initializing training files...done!
  -------------------------------------
  Request:  Metagenomic, Phase:  Gene Finding

  Error:  no input sequences to analyze.

Work dir:
  /home/chase/Documents/github/Autometa/work/d1/f8a46cbc7cd7f57bce035f5e0d8bba

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
evanroyrees commented 2 years ago

Splitting was deprecated for simplicity. This may be put back into the workflow for a later release, but for now this is no longer a problem.

chasemc commented 2 years ago

This dramatically increases the speed of the pre-binning steps of Autometa 2 (mostly by parallelizing prodigal) and the bug shouldn't be a difficult fix. Re-opening as an "enhancement"