TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm

[truelalin]

Hi, I built a transcriptome using rnaSpades and am trying to annotate it. Unfortunately, there was an error during the process that popped up:

Command failed: 'cat /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 40 -a /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep /home/lalini/anaconda3/envs/dammit/bin/hmmscan --cpu 1 --domtblout /dev/stdout -E 1e-05 -o /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep.x.pfam.tbl.hmmscan.out /home/lalini/.dammit//databases/Pfam-A.hmm /dev/stdin > /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep.x.pfam.tbl' returned 24

######################################## TaskFailed - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm hmmscan:longest_orfs.pep.x.Pfam-A.hmm : parallel: Warning: A record was longer than 1048576. Increasing to --blocksize 1363150. parallel: Warning: A record was longer than 1363150. Increasing to --blocksize 1772096. parallel: Warning: A record was longer than 1772096. Increasing to --blocksize 2303726. parallel: Warning: A record was longer than 2303726. Increasing to --blocksize 2994845. parallel: Warning: A record was longer than 2994845. Increasing to --blocksize 3893300.

hmmscan:longest_orfs.pep.x.Pfam-A.hmm :

I am a beginner in coding. Can anybody decipher this for me?

Thank you

[truelalin]

The station running it is 40 cores and I ran dammit at --n-threads 40. This is where I noticed the above problem first. Then, I went back removed sphinx installed by Anaconda and let dammit install it's own. Then tried running: dammit annotate transcriptome.fasta --database /.dammit/databases/ --busco-group fungi --n_threads 20 --user-databases /home/igor_projects/ data/P.sojae.pep.fasta

It's worked overnight and still on now at the cmscan:transcripts.fasta.x.Rfam.cm

Hopefully, dropping the number of threads was the solution.

[truelalin]

So it finished, after a day and a half..yeay! Unfortunately, BUSCO did not have fungi ODB9 installed properly.

Run Tasks

• [x] transcripts.fasta:
  • Python: function get_rename_transcriptome_task.fix
• [x] transcriptome_stats:transcripts.fasta:
  • Python: function get_transcriptome_stats_task.cmd
• [ ] busco:transcripts.fasta-fungi_odb9:
  • Cmd: python3 /home/lalini/anaconda3/envs/dammit/bin/run_BUSCO.py -i /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta -f -o transcripts.fasta.fungi.busco.results -l /home/lalini/.dammit//databases/busco2db/fungi_odb9 -m tran -c 20
• [x] TransDecoder.LongOrfs:transcripts.fasta:
  • Cmd: /home/lalini/anaconda3/envs/dammit/bin/TransDecoder.LongOrfs -t /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta -m 80
• [x] hmmscan:longest_orfs.pep.x.Pfam-A.hmm:
  • Cmd: cat /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 20 -a /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep /home/lalini/anaconda3/envs/dammit/bin/hmmscan --cpu 1 --domtblout /dev/stdout -E 1e-05 -o /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep.x.pfam.tbl.hmmscan.out /home/lalini/.dammit//databases/Pfam-A.hmm /dev/stdin > /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep.x.pfam.tbl
• [x] remap_hmmer:longest_orfs.pep.x.pfam.tbl:
  • Python: function get_remap_hmmer_task.cmd
• [x] hmmscan-gff3:transcripts.fasta.x.pfam-A.gff3:
  • Cmd: rm -f /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.pfam-A.gff3
  • Python: function get_hmmscan_gff3_task.cmd
• [x] TransDecoder.Predict:transcripts.fasta:
  • Cmd: /home/lalini/anaconda3/envs/dammit/bin/TransDecoder.Predict -t /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta --retain_pfam_hits /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder_dir/longest_orfs.pep.x.pfam.tbl
• [x] cmscan:transcripts.fasta.x.Rfam.cm:
  • Cmd: cat /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 20 -a /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta /home/lalini/anaconda3/envs/dammit/bin/cmscan --cpu 1 --rfam --nohmmonly -E 1e-05 --tblout /dev/stdout -o /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.rfam.tbl.cmscan.out /home/lalini/.dammit//databases/Rfam.cm /dev/stdin > /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.rfam.tbl
• [x] cmscan-gff3:transcripts.fasta.x.rfam.gff3:
  • Cmd: rm -f /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.rfam.gff3
  • Python: function get_cmscan_gff3_task.cmd
• [x] lastal:/home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.maf:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Cmd: cat /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 20 -a /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta /home/lalini/anaconda3/envs/dammit/bin/lastal -F15 -D100000.0 /home/lalini/.dammit//databases/aa_seq_euk.fasta > /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.maf
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] maf_best_hits:/home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.maf-/home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.best.csv:
  • Python: function get_maf_best_hits_task.cmd
• [x] maf-gff3:transcripts.fasta.x.OrthoDB.best.gff3:
  • Cmd: rm -f /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.best.gff3
  • Python: function get_maf_gff3_task.cmd
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-rename:/home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Python: rename_input
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-rename:/home/lalini/igor_projects/data/Phytophthora_sojae.P_sojae_V3_0.pep.all.fa:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Python: rename_input
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-translate:.transcripts.fasta:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Python: function translate_fastx
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-lastdb:.transcripts.fasta.pep:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Cmd: /home/lalini/anaconda3/envs/dammit/bin/lastdb -p -w3 .transcripts.fasta.pep .transcripts.fasta.pep
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-lastdb:.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Cmd: /home/lalini/anaconda3/envs/dammit/bin/lastdb -p -w3 .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-lastal:.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.x.transcripts.fasta.pep.maf:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Cmd: cat .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 20 -a .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa /home/lalini/anaconda3/envs/dammit/bin/lastal -D100000.0 .transcripts.fasta.pep > .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.x.transcripts.fasta.pep.maf
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-lastal:.transcripts.fasta.pep.x.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.maf:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Cmd: cat .transcripts.fasta.pep | /home/lalini/anaconda3/envs/dammit/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 20 -a .transcripts.fasta.pep /home/lalini/anaconda3/envs/dammit/bin/lastal -D100000.0 .Phytophthora_sojae.P_sojae_V3_0.pep.all.fa > .transcripts.fasta.pep.x.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.maf
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] user-database:Phytophthora_sojae.P_sojae_V3_0.pep.all.fa-shmlast-fit_and_filter_crbl_hits:
  • Python: function setup_profiler.add_profile_actions.start_profiling
  • Python: do_crbl_fit_and_filter
  • Python: function setup_profiler.add_profile_actions.stop_profiling
• [x] shmlast-gff3:transcripts.fasta.x.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.crbl.gff3:
  • Cmd: rm -f /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.crbl.gff3
  • Python: function get_shmlast_gff3_task.cmd
• [x] gff3-merge:transcripts.fasta.dammit.gff3:
  • Cmd: echo "##gff-version 3.2.1" > /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.dammit.gff3; cat /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.pfam-A.gff3 /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.transdecoder.gff3 /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.rfam.gff3 /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.OrthoDB.best.gff3 /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.x.Phytophthora_sojae.P_sojae_V3_0.pep.all.fa.crbl.gff3 | sed '/^#/ d' | sort | sed '/^$/d' >> /home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.dammit.gff3
• [x] fasta-annotate:/home/lalini/igor_projects/transcripts.fasta.dammit/transcripts.fasta.dammit.fasta:
  • Python: function get_annotate_fasta_task.annotate_fasta

Now let's do that...

[truelalin]

Busco was missing the ODB9 fungi database, I had to download and untar manually, not through BUSCO. After this, everything has worked and all the tasks were accomplished.