search

Dfam-consortium / RepeatModeler

De-Novo Repeat Discovery Tool
Other
194 stars 22 forks source link

problem installing RMBlast #89

Open DNAcastigator opened 4 years ago

DNAcastigator commented 4 years ago

Good morning, I'm trying to install repeatModeler and all its dependencies; I've so far compiled/installed RepeatMasker (with HMMER), RECON, RepeatScout, and TRF but I'm having trouble with RMBast: I'm following the exact instructions to compile from source as posted on their site (http://www.repeatmasker.org/RMBlast.html), but the compilation/installation fails when I run "make install":

if test -d /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/bin/gbench; then \
        make  install-gbench; \
    else \
        make  install-toolkit; \
    fi
make[1]: Entering directory `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++'
/bin/rm -rf /cm/shared/apps/repeatmodeler/RMBlast/include/ncbi-tools++
/usr/bin/install -c -d /cm/shared/apps/repeatmodeler/RMBlast/bin /cm/shared/apps/repeatmodeler/RMBlast/lib /cm/shared/apps/repeatmodeler/RMBlast/include/ncbi-tools++
/usr/bin/install -c /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/bin/* /cm/shared/apps/repeatmodeler/RMBlast/bin
/usr/bin/install -c -m 644 /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/*.* /cm/shared/apps/repeatmodeler/RMBlast/lib
/usr/bin/install: cannot stat `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/*.*': No such file or directory
make[1]: *** [install-toolkit] Error 1
make[1]: Leaving directory `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++'
make: *** [install] Error 2

It looks as if the patch they instruct users to apply on the NCBI Blast+ code is broken. may that be the problem?

jebrosen commented 4 years ago

Did you ensure to run make before running make install? That's the usual reason for seeing /usr/bin/install: cannot stat `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/*.*': No such file or directory

DNAcastigator commented 4 years ago

yes, I ran "make" before "make install," as is usual in any configure/make/make-install compilation. All of the other software that are provided have worked perfectly so far; it's only RMBlast that fails for some strange reason. Could it be that the lib path (ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/.') is incorrectly set in the Makefile, since the contents of lib after running "make" is a single directory (which does not conform to the pattern .).

In terms of compilers, I'm using GCC 6.3.0.

jebrosen commented 4 years ago

Could it be that the lib path (ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/.') is incorrectly set in the Makefile, since the contents of lib after running "make" is a single directory (which does not conform to the pattern .).

I doubt that that is the specific problem, because this same set of steps has worked reliably across several versions of GCC and several base operating systems. Can you post the full output of ./configure / make / make install? Perhaps there is a warning or error buried somewhere in the output.

DNAcastigator commented 4 years ago

configure.txt make.txt make-install.txt I attached here the requested output, I hope the mystery will be solved

jebrosen commented 4 years ago

make-install.txt:

 if test -d /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/bin/gbench; then \
        make  install-gbench; \
    else \
        make  install-toolkit; \
    fi
make[1]: Entering directory `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++'
/bin/rm -rf /cm/shared/apps/repeatmodeler/RMBlast/include/ncbi-tools++
/usr/bin/install -c -d /cm/shared/apps/repeatmodeler/RMBlast/bin /cm/shared/apps/repeatmodeler/RMBlast/lib /cm/shared/apps/repeatmodeler/RMBlast/include/ncbi-tools++
/usr/bin/install -c /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/bin/* /cm/shared/apps/repeatmodeler/RMBlast/bin
/usr/bin/install -c -m 644 /root/repeatmodeler/ncbi-blast-2.10.0+-src/c++/ReleaseMT/lib/*.* /cm/shared/apps/repeatmodeler/RMBlast/lib
make[1]: Leaving directory `/root/repeatmodeler/ncbi-blast-2.10.0+-src/c++'

I don't see any errors; was this a successful run? Or maybe stderr was not captured in these logs?

DNAcastigator commented 4 years ago

I'm sorry, my bad, here are the correct files configure.txt make.txt make-install.txt

jebrosen commented 4 years ago

make.txt looks pretty empty; maybe because sine you "built" it already there is nothing for make to do. Would you mind renaming/moving ReleaseMT and trying again so that the logs are from a clean run? Sorry for the inconvenience.

Now, this might be a problem that can be ignored. The RMBlast binary distribution we ship only includes the binaries, not the libraries, anyway. However, whatever is causing the lib/ directory to be empty for you could also have caused problems elsewhere - so I will keep trying to reproduce this problem on other machines.

DNAcastigator commented 4 years ago

thank you, unfortunately, the logs I sent earlier were from a fresh build; that is, I literally untarred NCBI Blast+ again and did the config/mak/make-install all over again precisely so there is nothing built before. The output of make looks that sparse from a fresh build. So, could I use the tool even with this problem?

jebrosen commented 4 years ago

I compared the files more carefully to my own logs and found an error snuck into your make output: /bin/sh: line 12: /usr/bin/sort: No such file or directory. This step is building a list of the directories that need to be compiled, which is why nothing is being compiled.

What operating system and version are you using? It is very strange that /usr/bin/sort is missing, unless maybe you only have /bin/sort and NCBI's build system is making an incorrect assumption. If you can figure out where sort is on your system, you should be able to edit the file c++/src/build-system/Makefile.meta_p and change the hardcoded /usr/bin/sort - e.g.

        /bin/sort -u $(PATTERN_FILTER) > $(RELEVANT_SUB_FILE)

before running configure / make / make install.

So, could I use the tool even with this problem?

If you still can't compile RMBlast yourself on that machine, you will have to get it another way. Our precompiled binaries are available at http://www.repeatmasker.org/RMBlast.html, and RMBlast is in a few package repositories such as bioconda. Many users have had issues with RepeatModeler from bioconda, but I don't know of any issues with their RMBlast.

Older versions of RepeatModeler also supported AB-Blast as the search engine, but we had some issues with it for RepeatModeler 2.0. We plan to investigate it more closely and hopefully restore compatibility in a future release.

DNAcastigator commented 4 years ago

The path for "sort" was indeed different from the one on my server (/bin/sort). Changing that line did the trick and it seems I was able to compile and install RMBlast. Many thank for all the help you provided and I'm sorry it took so much time.

jebrosen commented 4 years ago

Many thank for all the help you provided and I'm sorry it took so much time.

No trouble at all! I will send a bug report to NCBI so they are aware of this issue and can fix it in a future release.

DNAcastigator commented 4 years ago

I'm sorry to bother you again but I managed to install RM now and everything works fine; I installed also the components for LTR pipeline, but the RM configure script dies when I try to add their paths, specifically when I try to add CD-HIT's path:

The path to the installation of the CD-Hit sequence clustering package. CDHIT_DIR: /cm/shared/apps/cd-hit/4.8.1/bin Died at /cm/shared/apps/repeatmodeler/RepeatModeler-2.0.1/RepModelConfig.pm line 395, line 15.

jebrosen commented 4 years ago

Line 395 is when it writes the configuration file; it looks like there is not currently a nicer error message for that. This may be a permissions issue: the user running configure needs write access to RepModelConfig.pm.

DNAcastigator commented 4 years ago

The configure script has written fine before and exited without errors, so I don't think is a permissions issue. It only started failing when doing the optional LTR part. Could it change anything if a typed the path manually?

jebrosen commented 4 years ago

The configure script has written fine before and exited without errors, so I don't think is a permissions issue. It only started failing when doing the optional LTR part. Could it change anything if a typed the path manually?

That shouldn't make a difference; the code that is failing runs whether or not the LTR dependencies are enabled. If you had the wrong path for CD-HIT, it would have asked for the CD-HIT path again until it could find the expected programs.