search

oushujun / LTR_retriever

LTR_retriever is a highly accurate and sensitive program for identification of LTR retrotransposons; The LTR Assembly Index (LAI) is also included in this package.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5813529/
GNU General Public License v3.0
179 stars 40 forks source link

Unable to run LTR_retriever #15

Closed dipsaha closed 6 years ago

dipsaha commented 6 years ago

I am trying to run LTR_retriever from the ltr_finder results using the following scripts:

ltr_finder -D 15000 -d 1000 -L 7000 -l 100 -p 20 -C -M 0.9 /home/icar/Jute_genomes/ltr_finder/LLWS01.1.fa > /home/icar/Jute_genomes/ltr_finder/LTR_retriever_out/jro_524_genome.finder.scn

and then

perl /home/icar/Programs/LTR_retriever-master/LTR_retriever -genome /home/icar/Jute_genomes/ltr_finder/LLWS01.1.fa -infinder /home/icar/Jute_genomes/ltr_finder/LTR_retriever_out/jro_524_genome.finder.scn

but received the following error:

##########################

LTR_retriever v1.8.0

##########################

Contributors: Shujun Ou, Ning Jiang

Please cite: Ou S, Jiang N: LTR_retriever: a highly accurate and sensitive program for identification of long terminal repeat retrotransposons. Plant Physiology 2018, 176:1410-1422 Parameters: -genome /home/icar/Jute_genomes/ltr_finder/LLWS01.1.fa -infinder /home/icar/Jute_genomes/ltr_finder/LTR_retriever_out/jro_524_genome.finder.scn

Thu May 17 12:56:14 IST 2018 Dependency checking: The RMblast engine is not installed in RepeatMasker!

Tough I have configured Repeatmasker and received the following message, I am getting the above error while running LTR_retriever: Add a Search Engine:

  1. CrossMatch: [ Un-configured ]

  2. RMBlast - NCBI Blast with RepeatMasker extensions: [ Configured ]

  3. WUBlast/ABBlast (required by DupMasker): [ Un-configured ]

  4. HMMER3.1 & DFAM: [ Configured, Default ]

  5. Done

Enter Selection: 5 -- Setting perl interpreter...

Congratulations! RepeatMasker is now ready to use. The program is installed with a minimal repeat library by default. This library only contains simple, low-complexity, and common artefact ( contaminate ) sequences. These are adequate for use with your own custom repeat library. If you plan to search using common species specific repeats you will need to obtain the complete RepeatMasker repeat library from GIRI ( www.giriinst.org ) and install it in /home/icar/Programs/RepeatMasker.

Further documentation on the program may be found here: /home/icar/Programs/RepeatMasker/repeatmasker.help

Please help me to resolve the above problem in running the LTR_retriever. I am using Ubuntu 14.04 LTS 64 bit Thanks

oushujun commented 6 years ago

Hello,

Please put the genome and the input file in the same folder and run LTR_retriever within. If unsuccessful, please try these commands and get back to me: cp /path/LTR_retriever/database/dummy060817.fa ./dummy060817.test.fa /path/RepeatMasker -e ncbi -q -pa 1 -no_is -norna -nolow dummy060817.test.fa -lib dummy060817.test.fa

Thanks, Shujun

dipsaha commented 6 years ago

Hi oushujun,

I have re-installed the latest RepeatMasker in my /usr/local dir and updated RepLibrary with 'RepBaseRepeatMaskerEdition-20170127' follwed by reconfiguring the RepeatMasker.

Then I tried to run your commands: cp /path/LTR_retriever/database/dummy060817.fa ./dummy060817.test.fa /path/RepeatMasker -e ncbi -q -pa 1 -no_is -norna -nolow dummy060817.test.fa -lib dummy060817.test.fa

Now I get the following errors: In file: /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/ncResults-1526637556-27957.err Error: NCBI C++ Exception: "/home/icar/ncbi-rmblastn-2.2.28-src/c++/src/corelib/ncbiobj.cpp", line 925: Critical: ncbi::CObject::ThrowNullPointerException() - Attempt to access NULL pointer.

The command terminal showed follwing report:

icar@icar-crijaf:~/Programs/LTR_retriever-master$ cp /home/icar/Programs/LTR_retriever-master/database/dummy060817.fa ./dummy060817.test.fa icar@icar-crijaf:~/Programs/LTR_retriever-master$ /usr/local/RepeatMasker/RepeatMasker -e ncbi -q -pa 1 -no_is -norna -nolow dummy060817.test.fa -lib dummy060817.test.fa RepeatMasker version open-4.0.7 Search Engine: NCBI/RMBLAST [ 2.2.27+ ] Master RepeatMasker Database: /usr/local/RepeatMasker/Libraries/RepeatMaskerLib.embl ( Complete Database: dc20170127-rb20170127 ) Custom Repeat Library: dummy060817.test.fa

Building general libraries in: /home/icar/.RepeatMaskerCache/dc20170127-rb20170127/general

analyzing file dummy060817.test.fa identifying matches to dummy060817.test.fa sequences in batch 1 of 1 WARNING: The search engine returned an error (255, status = 255 ) Engine parameters: /home/icar/ncbi-rmblastn-2.2.28-src/c++/GCC480-Debug64/bin/rmblastn -db /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa -query /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa_batch-1.masked -gapopen 24 -gapextend 6 -mask_level 101 -complexity_adjust -word_size 11 -xdrop_ungap 450 -xdrop_gap_final 225 -xdrop_gap 112 -min_raw_gapped_score 225 -dust no -num_threads 4 -matrix 20p43g.matrix A search phase could not complete on this batch. The batch file will be re-run and if possible the program will resume. WARNING: Retrying batch ( 1 ) [ 255,, 111]... identifying matches to dummy060817.test.fa sequences in batch 1 of 1 WARNING: The search engine returned an error (255, status = 255 ) Engine parameters: /home/icar/ncbi-rmblastn-2.2.28-src/c++/GCC480-Debug64/bin/rmblastn -db /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa -query /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa_batch-1.masked -gapopen 24 -gapextend 6 -mask_level 101 -complexity_adjust -word_size 11 -xdrop_ungap 450 -xdrop_gap_final 225 -xdrop_gap 112 -min_raw_gapped_score 225 -dust no -num_threads 4 -matrix 20p43g.matrix A search phase could not complete on this batch. The batch file will be re-run and if possible the program will resume. WARNING: Retrying batch ( 1 ) [ 255,, 111]... identifying matches to dummy060817.test.fa sequences in batch 1 of 1 WARNING: The search engine returned an error (255, status = 255 ) Engine parameters: /home/icar/ncbi-rmblastn-2.2.28-src/c++/GCC480-Debug64/bin/rmblastn -db /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa -query /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018/dummy060817.test.fa_batch-1.masked -gapopen 24 -gapextend 6 -mask_level 101 -complexity_adjust -word_size 11 -xdrop_ungap 450 -xdrop_gap_final 225 -xdrop_gap 112 -min_raw_gapped_score 225 -dust no -num_threads 4 -matrix 20p43g.matrix A search phase could not complete on this batch. The batch file will be re-run and if possible the program will resume.

FATAL ERROR: RepeatMasker giving up. One or more batches failed! Unfortunately this type of error cannot be recovered from. Please submit the following details to the feedback page at the repeatmasker website:

   http://www.repeatmasker.org

RepeatMasker Version: open-4.0.7 Library Version: dc20170127-rb20170127 Search Engine: ncbi [ 2.2.27+ ] Command Line: /usr/local/RepeatMasker/RepeatMasker-e ncbi -q -pa 1 -no_is -norna -nolow dummy060817.test.fa -lib dummy060817.test.fa Batch Number: 1 Disk Space: Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 3778190784 1523243992 2063002704 43% /

System Memory: MemTotal: 65738344 kB MemFree: 42470636 kB Cached: 17393460 kB SwapCached: 0 kB SwapTotal: 66865148 kB SwapFree: 66865148 kB Further details about this problem may be found in the directory: /home/icar/Programs/LTR_retriever-master/RM_27941.FriMay181528592018

Let me know about your suggestions/resolutions to the above problem.

Thanks Dip

oushujun commented 6 years ago

Hello Dip,

Thank you for sharing the output info. I found two things that may go wrong.

  1. RepeatMasker is expecting the search engine NCBI/RMBLAST [ 2.2.27+ ], however, the 2.2.28 version seems to be given to the program. (/home/icar/ncbi-rmblastn-2.2.28-src/c++/GCC480-Debug64/bin/rmblastn). Please double check if the same version of BLAST+ and rmblast is installed.

  2. While I could not find anything wrong with the RepeatMasker Version: open-4.0.7, I had experience of unknown errors for higher version RepeatMasker (I recalled 4.0.4). So if you could not resolve the problem with reinstalling RMBLAST and reconfiguring RepeatMasker 4.0.7, you may try to install an older version of RepeatMasker. Currently I am using RepeatMasker v4.0.0 and RMBLAST 2.6.0. You may try these versions.

Thanks, Shujun

Weihankk commented 4 years ago

Hi Everyone, Today I meet the same error:

Mon Aug 3 14:10:12 CST 2020 Dependency checking: Error: The RMblast engine is not installed in RepeatMasker!

But I found that my taxonomy.dat is already gunzip done. And when I manuanlly run the RepeatMasker command I got the error message same with Dip:

FATAL ERROR: RepeatMasker giving up. One or more batches failed! Unfortunately this type of error cannot be recovered from. Please submit the following details to the feedback page at the repeatmasker website:

I install LTR_retriever by conda in a new environment. My LTR_retriever version is 2.9.0, my RepeatMasker version is 4.0.9, and my rmblast version is 2.2.28. So I try to reinstall rmblast in another version by conda:

conda install rmblast=2.6.0

Fortunately, the error was resolved. I run LTR_retriever successfully! Here I share my experience in solving this error, hoping to help others. :)

Weihan

AntetokounmJie commented 4 years ago

I download the lastest rmblast and repeatmasker. Then i congfigure it as the repeatmasker installation manul said. The ltr_retriever is runinng rightly.

I think the cause is the conda`s rmblast is too old.

oushujun commented 4 years ago

@Zea1nfO @Weihankk Thank you for your experiences. It is always tricky to get RepeatMasker/RepeatModeler working. I myself encounter these issues regularly.

Shujun