Error with link to libhts.a

NewtontheNeuron commented 2 weeks ago

Hello,

I have been trying to resolve an error for a long while now. It is similar to this issue submitted to htslib https://github.com/samtools/htslib/issues/688. I also submitted an issue and one of their devs shared that there may be an issue with cellsnp-lite's linking to the libhts.a file. Here is the issue I submitted.

Some details:

I am on a Linux HPC where I do not have route access
I am not able to use anaconda because the HPC that I am on does not support it
I installed htslib and libdeflate my self.
The location of libdeflate seems to be recognized by htslib when I run ldd libhts.so
I am using v1.20 of htslib and v1.2.3 of cellsnp-lite.
I have followed all the instructions online that involve --with-htslib, LDFLAGS="-Wl,-R$HOME/opt/htslib/lib", and etc.

I have added the error that I keep receiving when I run make.

Your assistance will be much appreciated.

/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/bgzf.c:602:(.text+0x1e86): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/bgzf.c:588:(.text+0x1ee6): undefined reference to `libdeflate_free_compressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `libdeflate_deflate':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1122:(.text+0x5e6): undefined reference to `libdeflate_alloc_compressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1137:(.text+0x659): undefined reference to `libdeflate_gzip_compress'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1138:(.text+0x665): undefined reference to `libdeflate_free_compressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1133:(.text+0x722): undefined reference to `libdeflate_free_compressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `uint7_decode_crc32':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:899:(.text+0x2ce2): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `uint7_decode_crc64':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:990:(.text+0x2db2): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `sint7_decode_crc32':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:944:(.text+0x2e82): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `itf8_decode_crc':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:251:(.text+0x495a): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:263:(.text+0x49bf): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o):/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:230: more undefined references to `libdeflate_crc32' follow
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `zlib_mem_inflate':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1068:(.text+0x61c8): undefined reference to `libdeflate_alloc_decompressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1085:(.text+0x6215): undefined reference to `libdeflate_gzip_decompress'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1102:(.text+0x622a): undefined reference to `libdeflate_free_decompressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1106:(.text+0x62cc): undefined reference to `libdeflate_free_decompressor'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `cram_read_block':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1423:(.text+0x63f1): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1425:(.text+0x642c): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `cram_write_block':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1542:(.text+0x6993): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1547:(.text+0x69bc): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o): in function `cram_uncompress_block':
/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:1581:(.text+0x6cda): undefined reference to `libdeflate_crc32'
/server/path/binutils-bin/2.41/ld: /home/newtonma/apps/htslib-1.20//lib/libhts.a(cram_io.o):/home/newtonma/dir/apps/htslib-1.20/cram/cram_io.c:4006: more undefined references to `libdeflate_crc32' follow
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:445: cellsnp-lite] Error 1
make[1]: Leaving directory '/lustre04/scratch/newtonma/brouillette_lab/apps/cellsnp-lite'
make: *** [Makefile:322: all] Error 2

hxj5 commented 2 weeks ago

Hi, we do recommend installing cellsnp-lite via conda/miniconda to avoid the dependency issues.

For manual installation as in your case, when I tried reproducing the error above, I found the installation of cellsnp-lite runs smoothly if building htslib without libdeflate (which should be the default behavior unless --with-libdeflate option is passed to ./configure in the htslib folder). Therefore, you may try re-building htslib to see whether it works without libdeflate.

I tested with the following environment ("libdeflate" is not installed, while other dependencies are available, such as "liblzma", "libbz2", "libcurl", "libcrypto"):

gcc (GCC) 4.8.5;
htslib 1.20 (source code downloaded from the official release htslib-1.20.tar.bz2). Building with default steps, i.e.,

autoreconf -i
./configure
make

cellsnp-lite 1.2.3 (source code cloned from the Github repo). Building with default steps (assuming the htslib folder is at ../htslib), i.e.,

autoreconf -iv
./configure
make

Alternatively, you can still use the current pre-built htslib. In this case, you may try removing (or renaming) the static library libhts.a to force linking cellsnp-lite to the shared htslib.so, as the build system of cellsnp-lite always uses the "libhts.a" when both libraries are detected.

NewtontheNeuron commented 1 week ago

Thank you. It took me several tries but it seems to have been built correctly now. Here is my process for anyone else who might have the same error.

I started again with a fresh install of htslib-1.20 and cellsnp-lite 1.2.3.
I made sure to name the folder containing htslib as htslib, and that it was located ../htslib relative to cellsnp-lite
I built both packages with the code provided above.
I did not run make install for htslib, which may have been causing path related problems
I changed the name of libhts.a
Double checked my .bashrc as well

single-cell-genetics / cellsnp-lite

Error with link to libhts.a #137