No Error error--skipping of TransDecoder.Predict:cdna_nointrons_utrs.fa cmd

[shannonekj]

dammit annotate cdna_nointrons_utrs.fa --user-databases pep.fa --busco-group eukaryota --n_threads 1

# dammit
## a tool for easy de novo transcriptome annotation

by Camille Scott

**v1.0.dev0**, 2016

## submodule: annotate
### Dependency Check

*All dependencies fulfilled.*

### Database Check
#### Info
* Database Directory: /Users/pliocene/.dammit/databases
* Doit Database: /Users/pliocene/.dammit/databases/databases.doit.db

*All database tasks up-to-date.*

### Annotation
#### Info
* Doit Database: /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/annotate.doit.db
* Input Transcriptome: /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa

Up-to-date tasks:
* BUSCO-eukaryota
* TransDecoder.LongOrfs
* cmscan:Rfam
* gff3:OrthoDB
* gff3:Pfam-A
* gff3:Rfam
* hmmscan:Pfam-A
* hmmscan:Pfam-A:remap
* lastal:OrthoDB
* lastal:best-hits:OrthoDB
* rename-transcriptome
* transcriptome-stats
* user-database:pep.fa-shmlast-lastal:.cdna_nointrons_utrs.fa.pep.x.pep.fa.maf
* user-database:pep.fa-shmlast-lastal:.pep.fa.x.cdna_nointrons_utrs.fa.pep.maf
* user-database:pep.fa-shmlast-lastdb:.cdna_nointrons_utrs.fa.pep
* user-database:pep.fa-shmlast-lastdb:.pep.fa
* user-database:pep.fa-shmlast-rename:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa
* user-database:pep.fa-shmlast-rename:/Users/pliocene/dammit_test/pep.fa
* user-database:pep.fa-shmlast-translate:.cdna_nointrons_utrs.fa

Out-of-date tasks:
* TransDecoder.Predict
* annotate:fasta
* gff3:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pep.fa.crbl.csv
* gff3:merge-all
* user-database:pep.fa-shmlast-fit_and_filter_crbl_hits

#### Run Tasks
- [x] cdna_nointrons_utrs.fa:
    * Python: function get_rename_transcriptome_task.fix
- [x] transcriptome_stats:cdna_nointrons_utrs.fa:
    * Python: function get_transcriptome_stats_task.cmd
- [x] busco:cdna_nointrons_utrs.fa-eukaryota:
    * Cmd: `python3 /Users/pliocene//BUSCO_v1.1b1/BUSCO_v1.1b1.py -in /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa -f -o cdna_nointrons_utrs.fa.eukaryota.busco.results -l /Users/pliocene/.dammit/databases/buscodb/eukaryota -m trans -c 1`
- [x] TransDecoder.LongOrfs:cdna_nointrons_utrs.fa:
    * Cmd: `/usr/local/bin/TransDecoder.LongOrfs -t /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa -m 80`
- [x] hmmscan:longest_orfs.pep.x.Pfam-A.hmm:
    * Cmd: `cat /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep | /usr/local/bin/parallel --progress --pipe -L 2 -N 400 --gnu -j 1 -a /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep /usr/local/bin/hmmscan --cpu 1 --domtblout /dev/stdout -E 1e-05 -o /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep.x.pfam.tbl.hmmscan.out /Users/pliocene/.dammit/databases/Pfam-A.hmm /dev/stdin > /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep.x.pfam.tbl`
- [x] remap_hmmer:longest_orfs.pep.x.pfam.tbl:
    * Python: function get_remap_hmmer_task.cmd
- [x] hmmscan-gff3:cdna_nointrons_utrs.fa.x.pfam-A.gff3:
    * Cmd: `rm -f /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pfam-A.gff3`
    * Python: function get_hmmscan_gff3_task.cmd
- [ ] TransDecoder.Predict:cdna_nointrons_utrs.fa:
    * Cmd: `/usr/local/bin/TransDecoder.Predict -t /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa --retain_pfam_hits /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep.x.pfam.tbl --retain_long_orfs 600`
- [x] cmscan:cdna_nointrons_utrs.fa.x.Rfam.cm:
    * Cmd: `cat /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa | /usr/local/bin/parallel --progress --pipe -L 2 -N 400 --gnu -j 1 -a /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa /usr/local/bin/cmscan --cpu 1 --rfam --nohmmonly -E 1e-05 --tblout /dev/stdout -o /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.rfam.tbl.cmscan.out /Users/pliocene/.dammit/databases/Rfam.cm /dev/stdin > /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.rfam.tbl`
- [x] cmscan-gff3:cdna_nointrons_utrs.fa.x.rfam.gff3:
    * Cmd: `rm -f /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.rfam.gff3`
    * Python: function get_cmscan_gff3_task.cmd
- [x] lastal:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.maf:
    * Cmd: `cat /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa | /usr/local/bin/parallel --progress --pipe -L 2 -N 400 --gnu -j 1 -a /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa /Users/pliocene//last-658/src/lastal -F15 -D100000.0 /Users/pliocene/.dammit/databases/aa_seq_euk.fasta > /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.maf`
- [x] maf_best_hits:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.maf-/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.best.csv:
    * Python: function get_maf_best_hits_task.cmd
- [x] maf-gff3:cdna_nointrons_utrs.fa.x.OrthoDB.best.gff3:
    * Cmd: `rm -f /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.best.gff3`
    * Python: function get_maf_gff3_task.cmd
- [x] user-database:pep.fa-shmlast-rename:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa:
    * Python: rename_input
- [x] user-database:pep.fa-shmlast-rename:/Users/pliocene/dammit_test/pep.fa:
    * Python: rename_input
- [x] user-database:pep.fa-shmlast-translate:.cdna_nointrons_utrs.fa:
    * Python: function translate_fastx
- [x] user-database:pep.fa-shmlast-lastdb:.cdna_nointrons_utrs.fa.pep:
    * Cmd: `/Users/pliocene//last-658/src/lastdb -p -w3 .cdna_nointrons_utrs.fa.pep .cdna_nointrons_utrs.fa.pep`
- [x] user-database:pep.fa-shmlast-lastdb:.pep.fa:
    * Cmd: `/Users/pliocene//last-658/src/lastdb -p -w3 .pep.fa .pep.fa`
- [x] user-database:pep.fa-shmlast-lastal:.pep.fa.x.cdna_nointrons_utrs.fa.pep.maf:
    * Cmd: `cat .pep.fa | /usr/local/bin/parallel --progress --pipe -L 2 -N 400 --gnu -j 1 -a .pep.fa /Users/pliocene//last-658/src/lastal -D100000.0 .cdna_nointrons_utrs.fa.pep > .pep.fa.x.cdna_nointrons_utrs.fa.pep.maf`
- [x] user-database:pep.fa-shmlast-lastal:.cdna_nointrons_utrs.fa.pep.x.pep.fa.maf:
    * Cmd: `cat .cdna_nointrons_utrs.fa.pep | /usr/local/bin/parallel --progress --pipe -L 2 -N 400 --gnu -j 1 -a .cdna_nointrons_utrs.fa.pep /Users/pliocene//last-658/src/lastal -D100000.0 .pep.fa > .cdna_nointrons_utrs.fa.pep.x.pep.fa.maf`
- [ ] user-database:pep.fa-shmlast-fit_and_filter_crbl_hits:
    * Python: do_crbl_fit_and_filter
/Users/pliocene/anaconda3/envs/py3.dammit/lib/python3.5/site-packages/shmlast-1.0-py3.5.egg/shmlast/crbl.py:61: RuntimeWarning: invalid value encountered in log10
  df[scaled_col_name] = -np.log10(df[scaled_col_name])
- [ ] shmlast-gff3:cdna_nointrons_utrs.fa.x.pep.fa.crbl.gff3:
    * Cmd: `rm -f /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pep.fa.crbl.gff3`
    * Python: function get_shmlast_gff3_task.cmd
- [ ] gff3-merge:cdna_nointrons_utrs.fa.dammit.gff3:
    * Cmd: `echo "##gff-version 3.2.1" > /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.dammit.gff3; cat /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pep.fa.crbl.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.best.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pfam-A.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.rfam.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder.gff3 | sed '/^#/ d' | sort | sed '/^$/d' >> /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.dammit.gff3`
- [ ] fasta-annotate:/Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.dammit.fasta:
    * Python: function get_annotate_fasta_task.annotate_fasta

[camillescott]

Not sure what the error is -- if you run it again after this, is TransDecoder run again?

[shannonekj]

Just running again it gets hung up on: `- [ ] TransDecoder.Predict:cdna_nointrons_utrs.fa:

• Cmd: /usr/local/bin/TransDecoder.Predict -t /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa --retain_pfam_hits /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep.x.pfam.tbl --retain_long_orfs 600`

Then, I ran dammit databases everything was noted as up-to-date. So I re-ran dammit annotate cdna_nointrons_utrs.fa --user-databases pep.fa --busco-group eukaryota --n_threads 1 and it showed Transdecoder.Predict as out of date with the Run Tasks skipping over

- [ ] TransDecoder.Predict:cdna_nointrons_utrs.fa:
    * Cmd: `/usr/local/bin/TransDecoder.Predict -t /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa --retain_pfam_hits /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder_dir/longest_orfs.pep.x.pfam.tbl --retain_long_orfs 600`

and

- [ ] gff3-merge:cdna_nointrons_utrs.fa.dammit.gff3:
    * Cmd: `echo "##gff-version 3.2.1" > /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.dammit.gff3; cat /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.rfam.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.transdecoder.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pfam-A.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.pep.fa.crbl.gff3 /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.x.OrthoDB.best.gff3 | sed '/^#/ d' | sort | sed '/^$/d' >> /Users/pliocene/dammit_test/cdna_nointrons_utrs.fa.dammit/cdna_nointrons_utrs.fa.dammit.gff3`

(perhaps worth note: the Transdecoder.Predict is coming from the 3.0.1 version of transdecoder)

[camillescott]

Fixed; if you do git pull; make install all should be good. Basically, in TransDecoder 3.0, they removed one of the output files; the TransDecoder task was expecting that file to exist, so in this case, it just never considered the task complete. Also note that the empty checkbox does not mean it was skipped, but rather that it was run (ie, all the other tasks were skipped, because they completed previously).

[shannonekj]

best thing* of the morning, thanks!

[shannonekj]

*coffee not included