search

samtools / htslib

C library for high-throughput sequencing data formats
Other
814 stars 446 forks source link

Trouble building htslib-1.11.tar.bz2 #1243

Open moldach opened 3 years ago

moldach commented 3 years ago

I'm running into problems trying to build htslib-1.11.tar.bz2:

$ wget https://github.com/samtools/htslib/releases/download/1.11/htslib-1.11.tar.bz2
$ tar -xf htslib-1.11.tar.bz2
$ cd htslib-1.11
$ ./configure PREFIX=/project/M-mtgraovac182840/tools/htslib-1.11
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/project/M-mtgraovac182840/tools/htslib-1.11':
configure: error: C compiler cannot create executables
See `config.log' for more details

More details here

config.log

This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by HTSlib configure 1.11, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure PREFIX=/project/M-mtgraovac182840/tools/htslib-1.11

## --------- ##
## Platform. ##
## --------- ##

hostname = marc
uname -m = x86_64
uname -r = 4.18.0-193.28.1.el8_2.x86_64
uname -s = Linux
uname -v = #1 SMP Thu Oct 22 00:20:22 UTC 2020

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /home/moldach/.local/bin
PATH: /home/moldach/bin
PATH: /global/software/gcc/gcc-4.9.2/bin
PATH: /project/M-mtgraovac182840/miniconda3/bin
PATH: /project/M-mtgraovac182840/miniconda3/condabin
PATH: /project/M-mtgraovac182840/perl5-matt/bin
PATH: /project/M-mtgraovac182840/tools/bwa-mem2-2.1_x64-linux
PATH: /project/M-mtgraovac182840/tools/vcftools_0.1.13/bin
PATH: /project/M-mtgraovac182840/tools/minia-v2.0.7-bin-Linux/bin
PATH: /project/M-mtgraovac182840/tools/seqtk-1.3
PATH: /project/M-mtgraovac182840/tools/bedtools2/bin
PATH: /project/M-mtgraovac182840/tools/SPAdes-3.15.0-Linux/bin
PATH: /project/M-mtgraovac182840/tools/bamUtil-1.0.14/bin
PATH: /project/M-mtgraovac182840/tools/parallel-20201222/bin
PATH: /project/M-mtgraovac182840/tools/MToolBox/MToolBox
PATH: /project/M-mtgraovac182840/tools/htslib-1.10.2
PATH: /project/M-mtgraovac182840/tools/freebayes-master/vcflib
PATH: /project/M-mtgraovac182840/tools/bowtie2-2.4.2-sra-linux-x86_64
PATH: /project/M-mtgraovac182840/tools/samtools-1.3.1/bin
PATH: /project/M-mtgraovac182840/tools/R-4.0.3/bin
PATH: /project/M-mtgraovac182840/tools
PATH: /project/M-mtgraovac182840/tools/CAP3
PATH: /project/M-mtgraovac182840/tools/ncbi-blast-2.11.0/bin
PATH: /home/moldach/.local/bin
PATH: /home/moldach/bin
PATH: /global/software/Modules/4.6.0/bin
PATH: /usr/local/bin
PATH: /usr/bin
PATH: /usr/local/sbin
PATH: /usr/sbin

## ----------- ##
## Core tests. ##
## ----------- ##

configure:2227: checking for gcc
configure:2243: found /global/software/gcc/gcc-4.9.2/bin/gcc
configure:2254: result: gcc
configure:2483: checking for C compiler version
configure:2492: gcc --version >&5
gcc (GCC) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:2503: $? = 0
configure:2492: gcc -v >&5
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/global/software/gcc/gcc-4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/global/software/gcc/gcc-4.9.2/ --enable-languages=c,c++,fortran,lto --disable-multilib
Thread model: posix
gcc version 4.9.2 (GCC) 
configure:2503: $? = 0
configure:2492: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:2503: $? = 1
configure:2492: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:2503: $? = 1
configure:2523: checking whether the C compiler works
configure:2545: gcc    conftest.c  >&5
gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found
compilation terminated.
configure:2549: $? = 1
configure:2587: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "HTSlib"
| #define PACKAGE_TARNAME "htslib"
| #define PACKAGE_VERSION "1.11"
| #define PACKAGE_STRING "HTSlib 1.11"
| #define PACKAGE_BUGREPORT "samtools-help@lists.sourceforge.net"
| #define PACKAGE_URL "http://www.htslib.org/"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:2592: error: in `/project/M-mtgraovac182840/tools/htslib-1.11':
configure:2594: error: C compiler cannot create executables
See `config.log' for more details

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_env_CC_set=
ac_cv_env_CC_value=
ac_cv_env_CFLAGS_set=
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_PKG_CONFIG_LIBDIR_set=
ac_cv_env_PKG_CONFIG_LIBDIR_value=
ac_cv_env_PKG_CONFIG_PATH_set=set
ac_cv_env_PKG_CONFIG_PATH_value=/global/software/gcc/gcc-4.9.2/lib64/pkgconfig:/global/software/gcc/gcc-4.9.2/lib/pkgconfig
ac_cv_env_PKG_CONFIG_set=
ac_cv_env_PKG_CONFIG_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_prog_ac_ct_CC=gcc

## ----------------- ##
## Output variables. ##
## ----------------- ##

CC='gcc'
CFLAGS=''
CPP=''
CPPFLAGS=''
CRYPTO_LIBS=''
DEFS=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP=''
EXEEXT=''
GREP=''
LDFLAGS=''
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
OBJEXT=''
PACKAGE_BUGREPORT='samtools-help@lists.sourceforge.net'
PACKAGE_NAME='HTSlib'
PACKAGE_STRING='HTSlib 1.11'
PACKAGE_TARNAME='htslib'
PACKAGE_URL='http://www.htslib.org/'
PACKAGE_VERSION='1.11'
PATH_SEPARATOR=':'
PKG_CONFIG=''
PKG_CONFIG_LIBDIR=''
PKG_CONFIG_PATH='/global/software/gcc/gcc-4.9.2/lib64/pkgconfig:/global/software/gcc/gcc-4.9.2/lib/pkgconfig'
PLATFORM=''
PLUGIN_EXT=''
RANLIB=''
SHELL='/bin/sh'
ac_ct_CC='gcc'
bindir='${exec_prefix}/bin'
build_alias=''
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
enable_plugins=''
exec_prefix='NONE'
gcs=''
host_alias=''
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
libcurl=''
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
oldincludedir='/usr/include'
pc_requires=''
pdfdir='${docdir}'
plugindir=''
pluginpath=''
prefix='NONE'
private_LIBS=''
program_transform_name='s,x,x,'
psdir='${docdir}'
runstatedir='${localstatedir}/run'
s3=''
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
static_LDFLAGS=''
static_LIBS=''
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "HTSlib"
#define PACKAGE_TARNAME "htslib"
#define PACKAGE_VERSION "1.11"
#define PACKAGE_STRING "HTSlib 1.11"
#define PACKAGE_BUGREPORT "samtools-help@lists.sourceforge.net"
#define PACKAGE_URL "http://www.htslib.org/"

configure: exit 77

Note: There are 4 different versions of gcc supplied on the system and I get the same error trying each of them

$ module avail gcc
---------------------------------- /global/software/Modules/4.6.0/modulefiles ----------------------------------
gcc/4.9.2  gcc/5.3.0  gcc/6.1.0  gcc/7.3.0

Trying newest version available: gcc/7.3.0

$ module load gcc/7.3.0
$ $ ./configure PREFIX=/project/M-mtgraovac182840/tools/htslib-1.11
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in `/project/M-mtgraovac182840/tools/htslib-1.11':
configure: error: C compiler cannot create executables
See `config.log' for more details
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.

It was created by HTSlib configure 1.11, which was
generated by GNU Autoconf 2.69.  Invocation command line was

  $ ./configure PREFIX=/project/M-mtgraovac182840/tools/htslib-1.11

## --------- ##
## Platform. ##
## --------- ##

hostname = marc
uname -m = x86_64
uname -r = 4.18.0-193.28.1.el8_2.x86_64
uname -s = Linux
uname -v = #1 SMP Thu Oct 22 00:20:22 UTC 2020

/usr/bin/uname -p = x86_64
/bin/uname -X     = unknown

/bin/arch              = x86_64
/usr/bin/arch -k       = unknown
/usr/convex/getsysinfo = unknown
/usr/bin/hostinfo      = unknown
/bin/machine           = unknown
/usr/bin/oslevel       = unknown
/bin/universe          = unknown

PATH: /global/software/gcc/gcc-7.3.0/bin
PATH: /home/moldach/.local/bin
PATH: /home/moldach/bin
PATH: /global/software/gcc/gcc-4.9.2/bin
PATH: /project/M-mtgraovac182840/miniconda3/bin
PATH: /project/M-mtgraovac182840/miniconda3/condabin
PATH: /project/M-mtgraovac182840/perl5-matt/bin
PATH: /project/M-mtgraovac182840/tools/bwa-mem2-2.1_x64-linux
PATH: /project/M-mtgraovac182840/tools/vcftools_0.1.13/bin
PATH: /project/M-mtgraovac182840/tools/minia-v2.0.7-bin-Linux/bin
PATH: /project/M-mtgraovac182840/tools/seqtk-1.3
PATH: /project/M-mtgraovac182840/tools/bedtools2/bin
PATH: /project/M-mtgraovac182840/tools/SPAdes-3.15.0-Linux/bin
PATH: /project/M-mtgraovac182840/tools/bamUtil-1.0.14/bin
PATH: /project/M-mtgraovac182840/tools/parallel-20201222/bin
PATH: /project/M-mtgraovac182840/tools/MToolBox/MToolBox
PATH: /project/M-mtgraovac182840/tools/htslib-1.10.2
PATH: /project/M-mtgraovac182840/tools/freebayes-master/vcflib
PATH: /project/M-mtgraovac182840/tools/bowtie2-2.4.2-sra-linux-x86_64
PATH: /project/M-mtgraovac182840/tools/samtools-1.3.1/bin
PATH: /project/M-mtgraovac182840/tools/R-4.0.3/bin
PATH: /project/M-mtgraovac182840/tools
PATH: /project/M-mtgraovac182840/tools/CAP3
PATH: /project/M-mtgraovac182840/tools/ncbi-blast-2.11.0/bin
PATH: /home/moldach/.local/bin
PATH: /home/moldach/bin
PATH: /global/software/Modules/4.6.0/bin
PATH: /usr/local/bin
PATH: /usr/bin
PATH: /usr/local/sbin
PATH: /usr/sbin

## ----------- ##
## Core tests. ##
## ----------- ##

configure:2227: checking for gcc
configure:2243: found /global/software/gcc/gcc-4.9.2/bin/gcc
configure:2254: result: gcc
configure:2483: checking for C compiler version
configure:2492: gcc --version >&5
gcc (GCC) 4.9.2
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:2503: $? = 0
configure:2492: gcc -v >&5
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/global/software/gcc/gcc-4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/global/software/gcc/gcc-4.9.2/ --enable-languages=c,c++,fortran,lto --disable-multilib
Thread model: posix
gcc version 4.9.2 (GCC) 
configure:2503: $? = 0
configure:2492: gcc -V >&5
gcc: error: unrecognized command line option '-V'
gcc: fatal error: no input files
compilation terminated.
configure:2503: $? = 1
configure:2492: gcc -qversion >&5
gcc: error: unrecognized command line option '-qversion'
gcc: fatal error: no input files
compilation terminated.
configure:2503: $? = 1
configure:2523: checking whether the C compiler works
configure:2545: gcc    conftest.c  >&5
gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found
compilation terminated.
configure:2549: $? = 1
configure:2587: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "HTSlib"
| #define PACKAGE_TARNAME "htslib"
| #define PACKAGE_VERSION "1.11"
| #define PACKAGE_STRING "HTSlib 1.11"
| #define PACKAGE_BUGREPORT "samtools-help@lists.sourceforge.net"
| #define PACKAGE_URL "http://www.htslib.org/"
| /* end confdefs.h.  */
| 
| int
| main ()
| {
| 
|   ;
|   return 0;
| }
configure:2592: error: in `/project/M-mtgraovac182840/tools/htslib-1.11':
configure:2594: error: C compiler cannot create executables
See `config.log' for more details

## ---------------- ##
## Cache variables. ##
## ---------------- ##

ac_cv_env_CC_set=
ac_cv_env_CC_value=
ac_cv_env_CFLAGS_set=
ac_cv_env_CFLAGS_value=
ac_cv_env_CPPFLAGS_set=
ac_cv_env_CPPFLAGS_value=
ac_cv_env_CPP_set=
ac_cv_env_CPP_value=
ac_cv_env_LDFLAGS_set=
ac_cv_env_LDFLAGS_value=
ac_cv_env_LIBS_set=
ac_cv_env_LIBS_value=
ac_cv_env_PKG_CONFIG_LIBDIR_set=
ac_cv_env_PKG_CONFIG_LIBDIR_value=
ac_cv_env_PKG_CONFIG_PATH_set=set
ac_cv_env_PKG_CONFIG_PATH_value=/global/software/gcc/gcc-7.3.0/lib64/pkgconfig:/global/software/gcc/gcc-7.3.0/lib/pkgconfig:/global/software/gcc/gcc-4.9.2/lib64/pkgconfig:/global/software/gcc/gcc-4.9.2/lib/pkgconfig
ac_cv_env_PKG_CONFIG_set=
ac_cv_env_PKG_CONFIG_value=
ac_cv_env_build_alias_set=
ac_cv_env_build_alias_value=
ac_cv_env_host_alias_set=
ac_cv_env_host_alias_value=
ac_cv_env_target_alias_set=
ac_cv_env_target_alias_value=
ac_cv_prog_ac_ct_CC=gcc

## ----------------- ##
## Output variables. ##
## ----------------- ##

CC='gcc'
CFLAGS=''
CPP=''
CPPFLAGS=''
CRYPTO_LIBS=''
DEFS=''
ECHO_C=''
ECHO_N='-n'
ECHO_T=''
EGREP=''
EXEEXT=''
GREP=''
LDFLAGS=''
LIBOBJS=''
LIBS=''
LTLIBOBJS=''
OBJEXT=''
PACKAGE_BUGREPORT='samtools-help@lists.sourceforge.net'
PACKAGE_NAME='HTSlib'
PACKAGE_STRING='HTSlib 1.11'
PACKAGE_TARNAME='htslib'
PACKAGE_URL='http://www.htslib.org/'
PACKAGE_VERSION='1.11'
PATH_SEPARATOR=':'
PKG_CONFIG=''
PKG_CONFIG_LIBDIR=''
PKG_CONFIG_PATH='/global/software/gcc/gcc-7.3.0/lib64/pkgconfig:/global/software/gcc/gcc-7.3.0/lib/pkgconfig:/global/software/gcc/gcc-4.9.2/lib64/pkgconfig:/global/software/gcc/gcc-4.9.2/lib/pkgconfig'
PLATFORM=''
PLUGIN_EXT=''
RANLIB=''
SHELL='/bin/sh'
ac_ct_CC='gcc'
bindir='${exec_prefix}/bin'
build_alias=''
datadir='${datarootdir}'
datarootdir='${prefix}/share'
docdir='${datarootdir}/doc/${PACKAGE_TARNAME}'
dvidir='${docdir}'
enable_plugins=''
exec_prefix='NONE'
gcs=''
host_alias=''
htmldir='${docdir}'
includedir='${prefix}/include'
infodir='${datarootdir}/info'
libcurl=''
libdir='${exec_prefix}/lib'
libexecdir='${exec_prefix}/libexec'
localedir='${datarootdir}/locale'
localstatedir='${prefix}/var'
mandir='${datarootdir}/man'
oldincludedir='/usr/include'
pc_requires=''
pdfdir='${docdir}'
plugindir=''
pluginpath=''
prefix='NONE'
private_LIBS=''
program_transform_name='s,x,x,'
psdir='${docdir}'
runstatedir='${localstatedir}/run'
s3=''
sbindir='${exec_prefix}/sbin'
sharedstatedir='${prefix}/com'
static_LDFLAGS=''
static_LIBS=''
sysconfdir='${prefix}/etc'
target_alias=''

## ----------- ##
## confdefs.h. ##
## ----------- ##

/* confdefs.h */
#define PACKAGE_NAME "HTSlib"
#define PACKAGE_TARNAME "htslib"
#define PACKAGE_VERSION "1.11"
#define PACKAGE_STRING "HTSlib 1.11"
#define PACKAGE_BUGREPORT "samtools-help@lists.sourceforge.net"
#define PACKAGE_URL "http://www.htslib.org/"

configure: exit 77

Additional information:

$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="8"
daviesrob commented 3 years ago

Thanks for the comprehensive report. It looks like this is the problem with your copies of gcc:

gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found

which I suspect may be related to this:

PATH: /project/M-mtgraovac182840/miniconda3/bin

It looks like you have a conda environment set up, which may be stopping your copies of gcc from finding the right shared libraries. If this is the case, you may find it easier to install htslib using bioconda. See #1125 for instructions, although to get htslib itself, you may want to change the last line to:

conda install 'htslib>=1.11'

Alternatively, you need to open a shell without starting the conda environment, and then try running ./configure again to see if gcc works without it. I would recommend using the latest one you have (7.3.0). If that works, then you should be able to build the library assuming the dependencies are all present.

moldach commented 3 years ago

Thanks @daviesrob

I've copied over my ~/.bashrc -> ~/.bashrc.bak and reset it (cp /etc/skel/.bashrc ~/.bashrc) so conda wont be activated. Next I shut down the terminal and logged back.

I'll try to do the installation again. The default version of gcc available is actually 8.3.1:

$ gcc --version
gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for ranlib... ranlib
checking for grep that handles long lines and -e... /usr/bin/grep
checking for C compiler warning flags... -Wall
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking shared library type for unknown-Linux... plain .so
checking whether the compiler accepts -fvisibility=hidden... yes
checking how to run the C preprocessor... gcc -E
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... yes
checking for getpagesize... yes
checking for working mmap... yes
checking for gmtime_r... yes
checking for fsync... yes
checking for drand48... yes
checking for srand48_deterministic... no
checking whether fdatasync is declared... yes
checking for fdatasync... yes
checking for library containing log... -lm
checking for zlib.h... yes
checking for inflate in -lz... yes
checking for library containing recv... none required
checking for bzlib.h... yes
checking for BZ2_bzBuffToBuffCompress in -lbz2... yes
checking for lzma.h... yes
checking for lzma_easy_buffer_encode in -llzma... yes
checking for libdeflate.h... no
checking for libdeflate_deflate_compress in -ldeflate... no
checking for curl_easy_pause in -lcurl... yes
checking for CCHmac... no
checking for library containing HMAC... -lcrypto
checking whether PTHREAD_MUTEX_RECURSIVE is declared... yes
configure: creating ./config.status
config.status: creating config.mk
config.status: creating htslib.pc.tmp
config.status: creating config.h
config.status: config.h is unchanged
$ make
sed -n '/^static_libs=/s/[^=]*=/HTSLIB_static_LIBS = /p;/^static_ldflags=/s/[^=]*=/HTSLIB_static_LDFLAGS = /p' htslib.pc.tmp > htslib_static.mk
sed -e 's#@-includedir@#'`pwd`'#g;s#@-libdir@#'`pwd`'#g' htslib.pc.tmp > htslib-uninstalled.pc
$ make install
mkdir -p -m 755 /project/M-mtgraovac182840/tools/htslib-1.11/bin /project/M-mtgraovac182840/tools/htslib-1.11/include /project/M-mtgraovac182840/tools/htslib-1.11/include/htslib /project/M-mtgraovac182840/tools/htslib-1.11/lib /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man1 /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man5 /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man7 /project/M-mtgraovac182840/tools/htslib-1.11/lib/pkgconfig
if test -n ""; then mkdir -p -m 755 ; fi
install -p -m 644 libhts.so /project/M-mtgraovac182840/tools/htslib-1.11/lib/libhts.so.1.11
ln -sf libhts.so.1.11 /project/M-mtgraovac182840/tools/htslib-1.11/lib/libhts.so
ln -sf libhts.so.1.11 /project/M-mtgraovac182840/tools/htslib-1.11/lib/libhts.so.3
sed -e 's#@-includedir@#/project/M-mtgraovac182840/tools/htslib-1.11/include#g;s#@-libdir@#/project/M-mtgraovac182840/tools/htslib-1.11/lib#g;s#@-PACKAGE_VERSION@#1.11#g' htslib.pc.tmp > /project/M-mtgraovac182840/tools/htslib-1.11/lib/pkgconfig/htslib.pc
chmod 644 /project/M-mtgraovac182840/tools/htslib-1.11/lib/pkgconfig/htslib.pc
install -p bgzip htsfile tabix /project/M-mtgraovac182840/tools/htslib-1.11/bin
if test -n ""; then install -p  ; fi
install -p -m 644 htslib/*.h /project/M-mtgraovac182840/tools/htslib-1.11/include/htslib
install -p -m 644 libhts.a /project/M-mtgraovac182840/tools/htslib-1.11/lib/libhts.a
install -p -m 644 bgzip.1 htsfile.1 tabix.1 /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man1
install -p -m 644 faidx.5 sam.5 vcf.5 /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man5
install -p -m 644 htslib-s3-plugin.7 /project/M-mtgraovac182840/tools/htslib-1.11/share/man/man7

Then I get an error from trying to launch bgzip:

$ bgzip -d human_g1k_v37.fasta.gz
bgzip: error while loading shared libraries: libbz2.so.1.0: cannot open shared object file: No such file or directory

If this is the case, you may find it easier to install htslib using bioconda Although I do have bioconda installed the process is actually a little more difficult than compiling from source because we have no outgoing internet connection on this server due to security concerns. So I need to build conda environments on another system than transfer those environments over.

moldach commented 3 years ago

I would like to determine if this is an underlying issue with gcc so I can let the Sys admins know (and close this issue), or if this is a problem with the build for the tool.

Thanks for your help

jkbonfield commented 3 years ago

checking for BZ2_bzBuffToBuffCompress in -lbz2... yes

Clearly the configure script found libbz2 on your system, so the config.log file should indicate where (it may take a bit of hunting).

$ bgzip -d human_g1k_v37.fasta.gz bgzip: error while loading shared libraries: libbz2.so.1.0: cannot open shared object file: No such file or directory

However this is implying otherwise. Some thoughts:

I can't explain otherwise why configure would detect it, gcc would compile and link it, but the run time in bash can no longer find the libraries it just built with.

daviesrob commented 3 years ago

There's something a bit odd about your set-up, as ./configure with no --prefix would normally try to install in /usr/local/. Or did you run ./configure again with --prefix?

Anyway, it looks like your gcc is finding libraries in locations that are not in the dynamic library search path. You can find where gcc looks by running:

gcc -print-search-dirs

The dynamic linker uses various methods to find libraries. You could look in /etc/ld.so.conf to see which directories are included in the cache maintained by ldconfig. You could also try running:

LD_DEBUG=libs,files /project/M-mtgraovac182840/tools/htslib-1.11/bin/bgzip

which will give you lots of information about where it's looking. If you look for libbz2.so in the output you should be able to find the interesting bit.

As a simple solution, if you find a directory with libbz2.so.1.0 in it in gcc's search path, you can add it to LD_LIBRARY_PATH to see if that solves the problem. If it does, you could try setting it as an RPATH when you configure htslib like this:

make distclean
./configure LDFLAGS='-W,-R/path'
make

(substituting /path with the one you actually need).