search

dib-lab / dammit

just annotate it, dammit!
http://dib-lab.github.io/dammit/
Other
88 stars 28 forks source link

HMM file error #40

Closed butterflyology closed 7 years ago

butterflyology commented 8 years ago

dammit seemed to run fine until it needed to run HMM. I received the following: "

Done preparing long ORFs.

Use file: Hmel.trans-0.3.fa.transdecoder_dir/longest_orfs.pep  for Pfam and/or BlastP searches to enable homology-based coding region identification.
Then, run TransDecoder.Predict for your final coding region predictions.
      [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm

Error: File format problem, trying to open HMM file /Users/Chris/.dammit/databases/Pfam-A.hmm. Opened /Users/Chris/.dammit/databases/Pfam-A.hmm.h3m, a pressed HMM file; but format of its .h3i file unrecognized [ ] cmscan:Hmel.trans-0.3.fa.x.Rfam.cm ^CSome tasks failed![dammit.annotate:ERROR] " The program then hung but did not exit. Running top showed that dammit was not using any of the CPU.

When I killed dammit here was the output:

" Error: File format problem, trying to open HMM file /Users/Chris/.dammit/databases/Pfam-A.hmm. Opened /Users/Chris/.dammit/databases/Pfam-A.hmm.h3m, a pressed HMM file; but format of its .h3i file unrecognized

      [ ] cmscan:Hmel.trans-0.3.fa.x.Rfam.cm

^CSome tasks failed![dammit.annotate:ERROR] TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR] Command failed: '/usr/local/bin/hmmscan --cpu 2 --domtblout Hmel.trans-0.3.fa.pfam-A.tbl -o Hmel.trans-0.3.fa.pfam-A.tbl.out /Users/Chris/.dammit/databases/Pfam-A.hmm Hmel.trans-0.3.fa.transdecoder_dir/longest_orfs.pep' returned 1 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:TransDecoder.Predict:Hmel.trans-0.3.fa[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Traceback (most recent call last): File "/Users/Chris/anaconda/bin/dammit", line 9, in app.DammitApp().run() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/app.py", line 31, in run self.args.func() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/app.py", line 183, in handle_annotate annotate.AnnotateHandler(self.args, db_handler.databases).handle() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/annotate.py", line 63, in handle self.run_tasks() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/annotate.py", line 82, in run_tasks common.run_tasks(self.tasks, doit_args, config=self.doit_config) File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/common.py", line 43, in run_tasks DoitMain(Loader()).run(args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/doit_cmd.py", line 168, in run return command.parse_execute(args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_base.py", line 122, in parse_execute return self.execute(params, args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_base.py", line 405, in execute return self._execute(**exec_params) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_run.py", line 253, in _execute return runner.run_all(self.control.task_dispatcher()) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 247, in run_all self.run_tasks(task_dispatcher) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 213, in run_tasks catched_excp = self.execute_task(node.task) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 175, in execute_task return task.execute(sys.stdout, sys.stderr, self.verbosity) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/task.py", line 390, in execute action_return = action.execute(task_stdout, task_stderr) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/action.py", line 211, in execute t_out.join() File "/Users/Chris/anaconda/lib/python2.7/threading.py", line 949, in join self.__block.wait() File "/Users/Chris/anaconda/lib/python2.7/threading.py", line 340, in wait waiter.acquire() KeyboardInterrupt "

camillescott commented 8 years ago

Maybe a HMMER version issue? Did you upgrade between building/installing the databases and running the annotations? On Dec 15, 2015 11:20 AM, "Chris Hamm" notifications@github.com wrote:

dammit seemed to run fine until it needed to run HMM. I received the following: " ################################# Done preparing long ORFs.

##################################

Use file: Hmel.trans-0.3.fa.transdecoder_dir/longest_orfs.pep for Pfam and/or BlastP searches to enable homology-based coding region identification. Then, run TransDecoder.Predict for your final coding region predictions. [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm

Error: File format problem, trying to open HMM file /Users/Chris/.dammit/databases/Pfam-A.hmm. Opened /Users/Chris/.dammit/databases/Pfam-A.hmm.h3m, a pressed HMM file; but format of its .h3i file unrecognized [ ] cmscan:Hmel.trans-0.3.fa.x.Rfam.cm ^CSome tasks failed![dammit.annotate:ERROR] " The program then hung but did not exit. Running top showed that dammit was not using any of the CPU.

When I killed dammit here was the output:

" Error: File format problem, trying to open HMM file /Users/Chris/.dammit/databases/Pfam-A.hmm. Opened /Users/Chris/.dammit/databases/Pfam-A.hmm.h3m, a pressed HMM file; but format of its .h3i file unrecognized

  [ ] cmscan:Hmel.trans-0.3.fa.x.Rfam.cm

^CSome tasks failed![dammit.annotate:ERROR] TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR] Command failed: '/usr/local/bin/hmmscan --cpu 2 --domtblout Hmel.trans-0.3.fa.pfam-A.tbl -o Hmel.trans-0.3.fa.pfam-A.tbl.out /Users/Chris/.dammit/databases/Pfam-A.hmm Hmel.trans-0.3.fa.transdecoder_dir/longest_orfs.pep' returned 1 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:TransDecoder.Predict:Hmel.trans-0.3.fa[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Traceback (most recent call last): File "/Users/Chris/anaconda/bin/dammit", line 9, in app.DammitApp().run() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/app.py", line 31, in run self.args.func() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/app.py", line 183, in handle_annotate annotate.AnnotateHandler(self.args, db_handler.databases).handle() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/annotate.py", line 63, in handle self.run_tasks() File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/annotate.py", line 82, in run_tasks common.run_tasks(self.tasks, doit_args, config=self.doit_config) File "/Users/Chris/anaconda/lib/python2.7/site-packages/dammit/common.py", line 43, in run_tasks DoitMain(Loader()).run(args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/doit_cmd.py", line 168, in run return command.parse_execute(args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_base.py", line 122, in parse_execute return self.execute(params, args) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_base.py", line 405, in execute return self._execute(**exec_params) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/cmd_run.py", line 253, in _execute return runner.run_all(self.control.task_dispatcher()) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 247, in run_all self.run_tasks(task_dispatcher) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 213, in run_tasks catched_excp = self.execute_task(node.task) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/runner.py", line 175, in execute_task return task.execute(sys.stdout, sys.stderr, self.verbosity) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/task.py", line 390, in execute action_return = action.execute(task_stdout, task_stderr) File "/Users/Chris/anaconda/lib/python2.7/site-packages/doit/action.py", line 211, in execute t_out.join() File "/Users/Chris/anaconda/lib/python2.7/threading.py", line 949, in join self.__block.wait() File "/Users/Chris/anaconda/lib/python2.7/threading.py", line 340, in wait waiter.acquire() KeyboardInterrupt "

— Reply to this email directly or view it on GitHub https://github.com/camillescott/dammit/issues/40.

butterflyology commented 8 years ago

Did that and trying again now.

butterflyology commented 8 years ago

Meant to comment on this earlier, sorry. All programs are up to date and I am still experiencing this issue.

#################################

Done preparing long ORFs.

##################################

Use file: short.Hmel.fa.transdecoder_dir/longest_orfs.pep  for Pfam and/or BlastP searches to enable homology-based coding region identification.

Then, run TransDecoder.Predict for your final coding region predictions.

      [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm

Error: File format problem, trying to open HMM file /Users/Chris/.dammit/databases/Pfam-A.hmm. Opened /Users/Chris/.dammit/databases/Pfam-A.hmm.h3m, a pressed HMM file; but format of its .h3i file unrecognized

      [ ] cmscan:short.Hmel.fa.x.Rfam.cm

      [ ] lastal:short.Hmel.fa.x.orthodb.maf

      [ ] maf_best_hits:short.Hmel.fa.x.orthodb.maf-
      short.Hmel.fa.x.orthodb.maf.best.csv

      [ ] maf-gff3:short.Hmel.fa.x.orthodb.maf.gff3

      [ ] cmscan-gff3:short.Hmel.fa.rfam.tbl.gff3

Some tasks failed![dammit.annotate:ERROR] TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR] Command failed: '/usr/local/bin/hmmscan --cpu 4 --domtblout short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl -E 1e-05 -o short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl.out /Users/Chris/.dammit/databases/Pfam-A.hmm short.Hmel.fa.transdecoder_dir/longest_orfs.pep' returned 1 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:remap_hmmer:longest_orfs.pep.pfam.tbl[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:TransDecoder.Predict:short.Hmel.fa[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3[dammit.annotate:ERROR] remap_hmmer:longest_orfs.pep.pfam.tbl [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:gff3-merge:short.Hmel.fa.dammit.gff3[dammit.annotate:ERROR] TransDecoder.Predict:short.Hmel.fa hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:fasta-annotate:short.Hmel.fa.dammit.fasta[dammit.annotate:ERROR] gff3-merge:short.Hmel.fa.dammit.gff3 [dammit.annotate:ERROR] Chris-MBP:dammit-test Chris$

butterflyology commented 8 years ago

I suspect this error is related to the one listed above, so I am just going to keep this thread alive.

Using version 0.2.5

Chris-MBP:dammit-test Chris$ dammit annotate short.Hmel.fa --busco-group arthropoda --n_threads 4

dammit! a tool for easy de novo transcriptome annotation

Camille Scott 2015

submodule: annotate

--- Checking PATH for dependencies

      [x] TransDecoder

      [x] LAST

      [x] HMMER

      [x] Infernal

      [x] crb-blast

      [x] BLAST+

      [x] BUSCO

--- Dependency results

      All dependencies satisfied!

--- Checking for database prep (dir: /Users/Chris/.dammit/databases)

      [x] download_and_gunzip:Pfam-A.hmm

      [x] hmmpress:Pfam-A.hmm

      [x] download_and_gunzip:Rfam.cm

      [x] cmpress:Rfam.cm

      [x] download_and_gunzip:aa_seq_euk.fasta

      [x] lastdb:aa_seq_euk.fasta.db

      [x] download_and_gunzip:ODB8_EukOGs_genes_ALL_levels.txt

      [x] download_and_untar:databases-arthropoda

--- Database results

      All databases prepared!

--- Running annotate!

      Transcriptome file: short.Hmel.fa

      Output directory:
      /Users/Chris/Desktop/Projects/Heliconius_transcriptome/melpomene/transcriptomes
      /dammit-test/short.Hmel.fa.dammit

      [ ] short.Hmel.fa

      [ ] transcriptome_stats:short.Hmel.fa

      [ ] busco:short.Hmel.fa-arthropoda

      [ ] transeq:short.Hmel.fa

      [ ] TransDecoder.LongOrfs:short.Hmel.fa

      [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm

      [ ] cmscan:short.Hmel.fa.x.Rfam.cm

      [ ] lastal:short.Hmel.fa.x.orthodb.maf

      [ ] maf_best_hits:short.Hmel.fa.x.orthodb.maf-
      short.Hmel.fa.x.orthodb.maf.best.csv

      [ ] maf-gff3:short.Hmel.fa.x.orthodb.maf.gff3

      [ ] cmscan-gff3:short.Hmel.fa.rfam.tbl.gff3

Some tasks failed![dammit.annotate:ERROR] TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR] Command failed: '/usr/local/bin/hmmscan --cpu 4 --domtblout short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl -E 1e-05 -o short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl.out /Users/Chris/.dammit/databases/Pfam-A.hmm short.Hmel.fa.transdecoder_dir/longest_orfs.pep' returned 1 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:remap_hmmer:longest_orfs.pep.pfam.tbl[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:TransDecoder.Predict:short.Hmel.fa[dammit.annotate:ERROR] hmmscan:longest_orfs.pep.x.Pfam-A.hmm [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3[dammit.annotate:ERROR] remap_hmmer:longest_orfs.pep.pfam.tbl [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:gff3-merge:short.Hmel.fa.dammit.gff3[dammit.annotate:ERROR] TransDecoder.Predict:short.Hmel.fa hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3 [dammit.annotate:ERROR] Some tasks failed![dammit.annotate:ERROR] UnmetDependency - taskid:fasta-annotate:short.Hmel.fa.dammit.fasta[dammit.annotate:ERROR] gff3-merge:short.Hmel.fa.dammit.gff3 [dammit.annotate:ERROR]

I followed Matt's advice and used "sudo -H pip install numexpr" which then changed the output and error to:

submodule: annotate


--- Checking PATH for dependencies

          [x] TransDecoder

          [x] LAST

          [x] HMMER

          [x] Infernal

          [x] crb-blast

          [x] BLAST+

          [x] BUSCO

--- Dependency results

          All dependencies satisfied!

--- Checking for database prep (dir: /Users/Chris/.dammit/databases)

          [x] download_and_gunzip:Pfam-A.hmm

          [x] hmmpress:Pfam-A.hmm

          [x] download_and_gunzip:Rfam.cm

          [x] cmpress:Rfam.cm

          [x] download_and_gunzip:aa_seq_euk.fasta

          [x] lastdb:aa_seq_euk.fasta.db

          [x] download_and_gunzip:ODB8_EukOGs_genes_ALL_levels.txt

          [x] download_and_untar:databases-arthropoda

--- Database results

          All databases prepared!

--- Running annotate!

          Transcriptome file: short.Hmel.fa

          Output directory:
          /Users/Chris/Desktop/Projects/Heliconius_transcriptome/melpomene/transcriptomes
          /dammit-test/short.Hmel.fa.dammit

          [x] short.Hmel.fa

          [x] transcriptome_stats:short.Hmel.fa

          [x] busco:short.Hmel.fa-arthropoda

          [x] transeq:short.Hmel.fa

          [x] TransDecoder.LongOrfs:short.Hmel.fa

          [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm

          [x] cmscan:short.Hmel.fa.x.Rfam.cm

          [x] lastal:short.Hmel.fa.x.orthodb.maf

          [x] maf_best_hits:short.Hmel.fa.x.orthodb.maf-
          short.Hmel.fa.x.orthodb.maf.best.csv

          [x] maf-gff3:short.Hmel.fa.x.orthodb.maf.gff3

          [x] cmscan-gff3:short.Hmel.fa.rfam.tbl.gff3

Some tasks failed![dammit.annotate:ERROR]
TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR]
Command failed: '/usr/local/bin/hmmscan --cpu 4 --domtblout short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl -E 1e-05 -o short.Hmel.fa.transdecoder_dir/longest_orfs.pep.pfam.tbl.out /Users/Chris/.dammit/databases/Pfam-A.hmm short.Hmel.fa.transdecoder_dir/longest_orfs.pep' returned 1
[dammit.annotate:ERROR]
Some tasks failed![dammit.annotate:ERROR]
UnmetDependency - taskid:remap_hmmer:longest_orfs.pep.pfam.tbl[dammit.annotate:ERROR]
hmmscan:longest_orfs.pep.x.Pfam-A.hmm
[dammit.annotate:ERROR]
Some tasks failed![dammit.annotate:ERROR]
UnmetDependency - taskid:TransDecoder.Predict:short.Hmel.fa[dammit.annotate:ERROR]
hmmscan:longest_orfs.pep.x.Pfam-A.hmm
[dammit.annotate:ERROR]
Some tasks failed![dammit.annotate:ERROR]
UnmetDependency - taskid:hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3[dammit.annotate:ERROR]
remap_hmmer:longest_orfs.pep.pfam.tbl
[dammit.annotate:ERROR]
Some tasks failed![dammit.annotate:ERROR]
UnmetDependency - taskid:gff3-merge:short.Hmel.fa.dammit.gff3[dammit.annotate:ERROR]
TransDecoder.Predict:short.Hmel.fa hmmscan-gff3:short.Hmel.fa.pfam.csv.gff3
[dammit.annotate:ERROR]
Some tasks failed![dammit.annotate:ERROR]
UnmetDependency - taskid:fasta-annotate:short.Hmel.fa.dammit.fasta[dammit.annotate:ERROR]
gff3-merge:short.Hmel.fa.dammit.gff3
[dammit.annotate:ERROR]
camillescott commented 7 years ago

Fixed during Beer Shoppe sitdown.