can't compile on CentOS 6.9

[esrice]

rust-htslib works great on all of my Ubuntu machines! Unfortunately, I need to get it to work on CentOS (6.9) because that's what the cluster I have access to runs. htslib will not compile due to some issue with clang. I installed the clang-devel package before building with cargo. Output is below. Any ideas you have about how to fix this would be wonderful. Thanks!

   Compiling rust-htslib v0.22.0                                                
error: failed to run custom build command for `rust-htslib v0.22.0`             
process didn't exit successfully: `/home/centos/trio_binning/target/release/build/rust-htslib-323139591f8804be/build-script-build` (exit code: 101)
--- stdout
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
echo '/* Default config.h generated by Makefile */' > config.h
echo '#define HAVE_LIBBZ2 1' >> config.h
echo '#define HAVE_LIBLZMA 1' >> config.h
echo '#define HAVE_LZMA_H 1' >> config.h
echo '#define HAVE_FSEEKO 1' >> config.h
echo '#define HAVE_DRAND48 1' >> config.h
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o kfunc.o kfunc.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o knetfile.o knetfile.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o kstring.o kstring.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o bcf_sr_sort.o bcf_sr_sort.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o bgzf.o bgzf.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o errmod.o errmod.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o faidx.o faidx.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o hfile.o hfile.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o hfile_net.o hfile_net.c
echo '#define HTS_VERSION "1.9"' > version.h
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o hts.o hts.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o hts_os.o hts_os.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o md5.o md5.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o multipart.o multipart.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o probaln.o probaln.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o realn.o realn.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o regidx.o regidx.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o sam.o sam.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o synced_bcf_reader.o synced_bcf_reader.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o vcf_sweep.o vcf_sweep.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o tbx.o tbx.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o textutils.o textutils.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o thread_pool.o thread_pool.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o vcf.o vcf.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o vcfutils.o vcfutils.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_codecs.o cram/cram_codecs.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_decode.o cram/cram_decode.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_encode.o cram/cram_encode.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_external.o cram/cram_external.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_index.o cram/cram_index.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_io.o cram/cram_io.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_samtools.o cram/cram_samtools.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/cram_stats.o cram/cram_stats.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/files.o cram/files.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/mFILE.o cram/mFILE.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/open_trace_file.o cram/open_trace_file.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/pooled_alloc.o cram/pooled_alloc.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/rANS_static.o cram/rANS_static.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/sam_header.o cram/sam_header.c
cc -O3 -ffunction-sections -fdata-sections -fPIC -m64 -static -I.  -c -o cram/string_alloc.o cram/string_alloc.c
ar -rc libhts.a kfunc.o knetfile.o kstring.o bcf_sr_sort.o bgzf.o errmod.o faidx.o hfile.o hfile_net.o hts.o hts_os.o md5.o multipart.o probaln.o realn.o regidx.o sam.o synced_bcf_reader.o vcf_sweep.o tbx.o textutils.o thread_pool.o vcf.o vcfutils.o cram/cram_codecs.o cram/cram_decode.o cram/cram_encode.o cram/cram_external.o cram/cram_index.o cram/cram_io.o cram/cram_samtools.o cram/cram_stats.o cram/files.o cram/mFILE.o cram/open_trace_file.o cram/pooled_alloc.o cram/rANS_static.o cram/sam_header.o cram/string_alloc.o
ranlib libhts.a
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
DEBUG = Some("false")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-static" "-o" "/home/centos/trio_binning/target/release/build/rust-htslib-610f77d1fd6e5b9f/out/wrapper.o" "-c" "wrapper.c"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
AR_x86_64-unknown-linux-gnu = None
AR_x86_64_unknown_linux_gnu = None
HOST_AR = None
AR = None
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
running: "ar" "crs" "/home/centos/trio_binning/target/release/build/rust-htslib-610f77d1fd6e5b9f/out/libwrapper.a" "/home/centos/trio_binning/target/release/build/rust-htslib-610f77d1fd6e5b9f/out/wrapper.o"
exit code: 0
TARGET = Some("x86_64-unknown-linux-gnu")
cargo:rustc-link-lib=static=wrapper
cargo:rustc-link-search=native=/home/centos/trio_binning/target/release/build/rust-htslib-610f77d1fd6e5b9f/out

--- stderr
thread 'main' panicked at 'function not loaded: clang_Type_getNumTemplateArguments', /home/centos/.cargo/registry/src/github.com-1ecc6299db9ec823/clang-sys-0.22.0/src/lib.rs:1470:1
note: Run with `RUST_BACKTRACE=1` for a backtrace.

[juliangehring]

This is likely caused by an old version of clang: CentOS 6.9 comes with clang 3.4 which is too old for some of the dependencies to build. I was able to reproduce your problem on CentOS 6 and 7. On CentOS 7, this all went away after upgrading from clang 3.4 to clang 5. Unfortunately, CentOS 6 doesn't seem to have newer versions of clang packaged - you would probably have to compile a new version by hand. A workaround might be to build a statically linked binary with musl (which is very easy to do with rust-htslib v0.22) on e.g. your ubuntu machine, and then run the binary on the "old" CentOS cluster.

[esrice]

Thanks so much for your response. I did manage to get a static binary working with musl, but I had to use commit 1f9736e to get that to work for some reason, or else I got the same error that your travis build gets:

https://travis-ci.org/rust-bio/rust-htslib/builds/456051357#L972

On Tue, Nov 20, 2018 at 3:36 PM Julian Gehring notifications@github.com wrote:

This is likely caused by an old version of clang: CentOS 6.9 comes with clang 3.4 which is too old for some of the dependencies to build. I was able to reproduce your problem on CentOS 6 and 7. On CentOS 7, this all went away after upgrading from clang 3.4 to clang 5. Unfortunately, CentOS 6 doesn't seem to have newer versions of clang packaged - you would probably have to compile a new version by hand https://www.vultr.com/docs/how-to-install-llvm-and-clang-on-centos-6. A workaround might be to build a statically linked binary with musl (which is very easy to do with rust-htslib v0.22) on e.g. your ubuntu machine, and then run the binary on the "old" CentOS cluster.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rust-bio/rust-htslib/issues/114#issuecomment-440420788, or mute the thread https://github.com/notifications/unsubscribe-auth/AFa-paxGTWpoG3c5tRF96BvhPvkYw00Vks5uxGfigaJpZM4YVOmz .

[juliangehring]

The support for musl has been added with v0.22, it might be easiest to build with this version directly.

[juliangehring]

Do you mean that building with the tagged v0.22.0 does not work, but building with https://github.com/rust-bio/rust-htslib/commit/1f9736ef108762e919802b3d78e22e47533c122c does?

[johanneskoester]

You can get a newer clang via Conda: https://anaconda.org/search?q=clang

[johanneskoester]

Bioconda builds on CentOS 6 using these packages.

[esrice]

That is correct; building musl-linked binaries with v0.22.0 results in the same bzlib error you get in your latest Travis build, but building with 1f9736e https://github.com/rust-bio/rust-htslib/commit/1f9736ef108762e919802b3d78e22e47533c122c works fine.

On Thu, Nov 22, 2018, 4:58 AM Julian Gehring <notifications@github.com wrote:

Do you mean that building with the tagged v0.22.0 does not work, but building with 1f9736e https://github.com/rust-bio/rust-htslib/commit/1f9736ef108762e919802b3d78e22e47533c122c does?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rust-bio/rust-htslib/issues/114#issuecomment-440976057, or mute the thread https://github.com/notifications/unsubscribe-auth/AFa-pV92s7-hw_U9bXiG2F6TKijdwb61ks5uxnVagaJpZM4YVOmz .

[juliangehring]

@esrice Maybe the same problem as in #115?

[esrice]

Looks like it, yes.

On Mon, Nov 26, 2018 at 1:53 PM Julian Gehring notifications@github.com wrote:

@esrice https://github.com/esrice Maybe the same problem as in #115 https://github.com/rust-bio/rust-htslib/pull/115?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rust-bio/rust-htslib/issues/114#issuecomment-441754197, or mute the thread https://github.com/notifications/unsubscribe-auth/AFa-pSTe_yPyBRlNCFcj6LWdEO1217z2ks5uzDjBgaJpZM4YVOmz .

[TyberiusPrime]

This might be a separate issue, but I can't use htslib in PyO3 'well', as in I can't make a good distribution out of them.

Python packages may be distributed as 'wheels', ie. binaries that are build in a 'manylinux' environment which is essentially an ancient CentOS 6 (as opposed to a source distribution which would require a nightly rust on the target machine).

This is the error I get when building my wheels

expr: warning: unportable BRE: `^\\([0-9.]*\\)$': using `^' as the first character
of the basic regular expression is not portable; it is being ignored
thread_pool.c: In function ‘hts_tpool_init’:
thread_pool.c:679:38: error: ‘PTHREAD_MUTEX_RECURSIVE’ undeclared (first use in this function)
     pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE);

(I'm building with a git htslib dependency).

[johanneskoester]

@TyberiusPrime what you see there is a compiler error from htslib itself. So, a fix for that has to be done in the htslib source. I suggest to file an issue at github.com/samtools/htslib.