search

rust-bio / rust-htslib

This library provides HTSlib bindings and a high level Rust API for reading and writing BAM files.
MIT License
308 stars 80 forks source link

hts-sys fails to compile on OSX #202

Closed mxygc closed 4 years ago

mxygc commented 4 years ago

Hi developers, I'm building an blank program (unmodified hello world main.rs) in order to see how the dependencies are being installed. Here is what I added to Cargo.toml:

[dependencies]
# rust-htslib = "*"

Here is my build result, including an error spit from the C compiler:

User1@localhost~/rust/bam_test$ cargo build
   Compiling cc v1.0.53
   Compiling pkg-config v0.3.17
   Compiling libc v0.2.70
   Compiling version_check v0.9.1
   Compiling glob v0.3.0
   Compiling memchr v2.3.3
   Compiling proc-macro2 v1.0.13
   Compiling bitflags v1.2.1
   Compiling unicode-xid v0.2.0
   Compiling regex-syntax v0.6.17
   Compiling bindgen v0.53.2
   Compiling quick-error v1.2.3
   Compiling rustc-hash v1.1.0
   Compiling lazycell v1.2.1
   Compiling shlex v0.1.1
   Compiling peeking_take_while v0.1.2
   Compiling cfg-if v0.1.10
   Compiling lazy_static v1.4.0
   Compiling fs-utils v1.1.4
   Compiling nom v5.1.1
   Compiling clang-sys v0.29.3
   Compiling quote v1.0.5
   Compiling libloading v0.5.2
   Compiling libz-sys v1.0.25
   Compiling curl-sys v0.4.31+curl-7.70.0
   Compiling lzma-sys v0.1.16
   Compiling bzip2-sys v0.1.8+1.0.8
   Compiling regex v1.3.7
   Compiling cexpr v0.4.0
   Compiling hts-sys v1.10.2
error: failed to run custom build command for `hts-sys v1.10.2`

Caused by:
  process didn't exit successfully: `/Users/User1/rust/bam_test/target/debug/build/hts-sys-1d2967724766e45e/build-script-build` (exit code: 101)
--- stdout
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = None
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
echo '/* Default config.h generated by Makefile */' > config.h
echo '#define HAVE_LIBBZ2 1' >> config.h
echo '#define HAVE_LIBLZMA 1' >> config.h
echo '#ifndef __APPLE__' >> config.h
echo '#define HAVE_LZMA_H 1' >> config.h
echo '#endif' >> config.h
echo '#define HAVE_DRAND48 1' >> config.h
echo '#define HAVE_LIBCURL 1' >> config.h
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o kfunc.o kfunc.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o knetfile.o knetfile.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o kstring.o kstring.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o bcf_sr_sort.o bcf_sr_sort.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o bgzf.o bgzf.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o errmod.o errmod.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o faidx.o faidx.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o header.o header.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o hfile.o hfile.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o hfile_net.o hfile_net.c
echo '#define HTS_VERSION_TEXT "1.10.2"' > version.h
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -static -I /Users/User1/rust/bam_test/target/debug/build/bzip2-sys-fd8f7cec15178433/out/include -I.  -c -o hts.o hts.c

--- stderr
In file included from hts.c:44:
In file included from ./cram/cram.h:46:
./header.h:227:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_fill_hrecs(sam_hdr_t *bh);
                       ^
./header.h:236:21: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_rebuild(sam_hdr_t *bh);
                    ^
./header.h:244:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_new(void);
^
./header.h:250:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
^
./header.h:250:28: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
                           ^
./header.h:259:34: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                 ^
./header.h:259:55: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                                      ^
./header.h:268:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_rebuild_text(const sam_hrecs_t *hrecs, kstring_t *ks);
                                 ^
./header.h:276:21: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
void sam_hrecs_free(sam_hrecs_t *hrecs);
                    ^
./header.h:285:41: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_type_t *sam_hrecs_find_type_id(sam_hrecs_t *hrecs, const char *type,
                                        ^
./header.h:292:26: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_remove_key(sam_hrecs_t *hrecs,
                         ^
./header.h:302:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_rg_t *sam_hrecs_find_rg(sam_hrecs_t *hrecs, const char *rg);
                                 ^
./header.h:305:42: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_sort_order sam_hrecs_sort_order(sam_hrecs_t *hrecs);
                                         ^
./header.h:308:44: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_group_order sam_hrecs_group_order(sam_hrecs_t *hrecs);
                                           ^
In file included from hts.c:44:
In file included from ./cram/cram.h:47:
./cram/cram_structs.h:683:5: error: must use 'struct' tag to refer to type 'sam_hdr_t'
    sam_hdr_t     *header;
    ^
    struct 
In file included from hts.c:44:
In file included from ./cram/cram.h:48:
./cram/cram_io.h:671:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int refs2id(refs_t *r, sam_hdr_t *hdr);
                       ^
                       struct 
./cram/cram_io.h:828:1: error: must use 'struct' tag to refer to type 'sam_hdr_t'
sam_hdr_t *cram_read_SAM_hdr(cram_fd *fd);
^
struct 
./cram/cram_io.h:836:37: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_write_SAM_hdr(cram_fd *fd, sam_hdr_t *hdr);
                                    ^
                                    struct 
./cram/cram_io.h:929:41: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_set_header2(cram_fd *fd, const sam_hdr_t *hdr);
                                        ^
                                        struct 
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [hts.o] Error 1
thread 'main' panicked at 'failed to build htslib', ./cargo/registry/src/github.com-1ecc6299db9ec823/hts-sys-1.10.2/build.rs:86:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Does anybody know how to sort it out? Thanks.

Update: rustc 1.43.1 (8d69840ab 2020-05-04) macOS 10.15.4

brainstorm commented 4 years ago

Would you mind trying with this PR/branch?: https://github.com/rust-bio/rust-htslib/pull/193

You'll have to use Rustembedded cross (docker cross-compiling wrapper)... let me know how it goes, looking for early beta testers on my (now working) PR :)

In a nutshell, you'd just have to run:

$ git clone --recursive https://github.com/brainstorm/rust-htslib
$ cargo install cross
$ cd rust-htslib && cross build

It works for me for GNU and MUSL toolchains :)

luyoutao commented 4 years ago

@brainstorm Well that doesn't explain why glibc fails.

brainstorm commented 4 years ago

@luyoutao You reckon there's some missing dependency/crate/flag somewhere? I'm all ears! :-S

luyoutao commented 4 years ago

@brainstorm must use 'struct' tag to refer to type 'sam_hdr_t' looks like a compiler complaint when missing some header. Just my gut feeling.

brainstorm commented 4 years ago

@foehnair Could you please try again cloning from master (and following the new README.md steps)? It should work right away ;)

mxygc commented 4 years ago

@brainstorm I tried the github version but still failed:

In file included from hts.c:44:
In file included from ./cram/cram.h:46:
./header.h:227:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_fill_hrecs(sam_hdr_t *bh);
                       ^
./header.h:236:21: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_rebuild(sam_hdr_t *bh);
                    ^
./header.h:244:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_new(void);
^
./header.h:250:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
^
./header.h:250:28: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
                           ^
./header.h:259:34: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                 ^
./header.h:259:55: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                                      ^
./header.h:268:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_rebuild_text(const sam_hrecs_t *hrecs, kstring_t *ks);
                                 ^
./header.h:276:21: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
void sam_hrecs_free(sam_hrecs_t *hrecs);
                    ^
./header.h:285:41: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_type_t *sam_hrecs_find_type_id(sam_hrecs_t *hrecs, const char *type,
                                        ^
./header.h:292:26: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_remove_key(sam_hrecs_t *hrecs,
                         ^
./header.h:302:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_rg_t *sam_hrecs_find_rg(sam_hrecs_t *hrecs, const char *rg);
                                 ^
./header.h:305:42: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_sort_order sam_hrecs_sort_order(sam_hrecs_t *hrecs);
                                         ^
./header.h:308:44: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_group_order sam_hrecs_group_order(sam_hrecs_t *hrecs);
                                           ^
In file included from hts.c:44:
In file included from ./cram/cram.h:47:
./cram/cram_structs.h:683:5: error: must use 'struct' tag to refer to type 'sam_hdr_t'
    sam_hdr_t     *header;
    ^
    struct 
In file included from hts.c:44:
In file included from ./cram/cram.h:48:
./cram/cram_io.h:671:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int refs2id(refs_t *r, sam_hdr_t *hdr);
                       ^
                       struct 
./cram/cram_io.h:828:1: error: must use 'struct' tag to refer to type 'sam_hdr_t'
sam_hdr_t *cram_read_SAM_hdr(cram_fd *fd);
^
struct 
./cram/cram_io.h:836:37: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_write_SAM_hdr(cram_fd *fd, sam_hdr_t *hdr);
                                    ^
                                    struct 
./cram/cram_io.h:929:41: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_set_header2(cram_fd *fd, const sam_hdr_t *hdr);
                                        ^
                                        struct 
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [hts.o] Error 1
thread 'main' panicked at 'failed to build htslib', hts-sys/build.rs:117:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Here is my clang version 

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.0 (clang-1100.0.33.17)
Target: x86_64-apple-darwin19.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Any thoughts? Thanks.

brainstorm commented 4 years ago

Which commands did you use to compile it?

mxygc commented 4 years ago

I used cargo build --verbose in the directory rust-htslib.

brainstorm commented 4 years ago

Ok, you should use the ones shown in the README instead.

mxygc commented 4 years ago

cross build # will build with GNU toolchain

I thought this was for GNU toolchain but I don't have GNU GCC.

brainstorm commented 4 years ago

Good point, fixed on https://github.com/rust-bio/rust-htslib/commit/b9ed3ad49fcd3e9af6fc282343876ee743f02156

mxygc commented 4 years ago

Nice. Here are results from cross build --verbose:

     Running `/Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-d8d36db1aff8a3bc/build-script-build`
error: failed to run custom build command for `hts-sys v1.10.2 (/Users/user/Workspace/misc/rust-htslib/hts-sys)`

Caused by:
  process didn't exit successfully: `/Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-d8d36db1aff8a3bc/build-script-build` (exit code: 101)
--- stdout
OPT_LEVEL = Some("0")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = None
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = Some("-I/usr/local/musl/include")
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
rm -f test/*.tmp test/*.tmp.* test/longrefs/*.tmp.* test/tabix/*.tmp.* test/tabix/FAIL* header-exports.txt shlib-exports-dylib.txt
rm -f *.o *.pico cram/*.o cram/*.pico test/*.o test/*.dSYM version.h
rm -f hts-object-files
rm -f libhts.dylib libhts.*.dylib
rm -f libhts.a bgzip htsfile tabix  test/hts_endian test/fieldarith test/hfile test/pileup test/sam test/test_bgzf test/test_kstring test/test_realn test/test-regidx test/test_str2int test/test_view test/test_index test/test-vcf-api test/test-vcf-sweep test/test-bcf-sr test/fuzz/hts_open_fuzzer.o test/test-bcf-translate test/test-parse-reg test/thrash_threads1 test/thrash_threads2 test/thrash_threads3 test/thrash_threads4 test/thrash_threads5 test/thrash_threads6 test/thrash_threads7
echo '/* Default config.h generated by Makefile */' > config.h
echo '#define HAVE_LIBBZ2 1' >> config.h
echo '#define HAVE_LIBLZMA 1' >> config.h
echo '#ifndef __APPLE__' >> config.h
echo '#define HAVE_LZMA_H 1' >> config.h
echo '#endif' >> config.h
echo '#define HAVE_DRAND48 1' >> config.h
echo '#define HAVE_LIBCURL 1' >> config.h
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o kfunc.o kfunc.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o knetfile.o knetfile.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o kstring.o kstring.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o bcf_sr_sort.o bcf_sr_sort.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o bgzf.o bgzf.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o errmod.o errmod.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o faidx.o faidx.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o header.o header.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o hfile.o hfile.c
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o hfile_net.o hfile_net.c
echo '#define HTS_VERSION_TEXT ""' > version.h
cc -ffunction-sections -fdata-sections -fPIC -g -fno-omit-frame-pointer -m64 -I/usr/local/musl/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/libz-sys-0b8ad0480df1b3ca/out/include -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/bzip2-sys-3e8b4c5b45bdce3b/out/include -I /Users/user/Applications/cargo/registry/src/github.com-1ecc6299db9ec823/lzma-sys-0.1.16/xz-5.2/src/liblzma/api -I /Users/user/Workspace/misc/rust-htslib/target/debug/build/curl-sys-bfce02cddd16dc9b/out/include -Wall -Wextra -I.  -c -o hts.o hts.c

--- stderr
fatal: not a git repository: /Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-2162d2f6293c6445/out/htslib/../../.git/modules/htslib
fatal: not a git repository: /Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-2162d2f6293c6445/out/htslib/../../.git/modules/htslib
fatal: not a git repository: /Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-2162d2f6293c6445/out/htslib/../../.git/modules/htslib
fatal: not a git repository: /Users/user/Workspace/misc/rust-htslib/target/debug/build/hts-sys-2162d2f6293c6445/out/htslib/../../.git/modules/htslib
knetfile.c:380:55: warning: unused parameter 'mode' [-Wunused-parameter]
knetFile *khttp_parse_url(const char *fn, const char *mode)
                                                      ^
knetfile.c:518:42: warning: unused parameter 'mode' [-Wunused-parameter]
knetFile *knet_dopen(int fd, const char *mode)
                                         ^
2 warnings generated.
kstring.c:158:12: warning: comparison of integers of different signs: 'int' and 'unsigned long' [-Wsign-compare]
        if (l + 1 > s->m - s->l) {
            ~~~~~ ^ ~~~~~~~~~~~
1 warning generated.
bcf_sr_sort.c:55:16: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
    for (i=0; i<dst->n; i++) dst->b[i] |= src->b[i];
              ~^~~~~~~
bcf_sr_sort.c:351:17: warning: comparison of integers of different signs: 'int' and 'khint_t' (aka 'unsigned int') [-Wsign-compare]
    for (k=0; k < kh_end(hash); k++)
              ~ ^ ~~~~~~~~~~~~
bcf_sr_sort.c:354:17: warning: comparison of integers of different signs: 'int' and 'khint_t' (aka 'unsigned int') [-Wsign-compare]
    for (k=0; k < kh_end(hash); k++)
              ~ ^ ~~~~~~~~~~~~
bcf_sr_sort.c:629:43: warning: unused parameter 'readers' [-Wunused-parameter]
void bcf_sr_sort_remove_reader(bcf_srs_t *readers, sr_sort_t *srt, int i)
                                          ^
4 warnings generated.
In file included from bgzf.c:46:
./htslib/hfile.h:271:35: warning: comparison of integers of different signs: 'long' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (fp->limit - fp->begin < nbytes){
            ~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~
bgzf.c:583:101: warning: unused parameter 'level' [-Wunused-parameter]
static int bgzf_gzip_compress(BGZF *fp, void *_dst, size_t *dlen, const void *src, size_t slen, int level)
                                                                                                    ^
bgzf.c:1113:23: warning: comparison of integers of different signs: 'ssize_t' (aka 'long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
    while (bytes_read < length) {
           ~~~~~~~~~~ ^ ~~~~~~
bgzf.c:1126:43: warning: comparison of integers of different signs: 'unsigned long' and 'int' [-Wsign-compare]
        copy_length = length - bytes_read < available? length - bytes_read : available;
                      ~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~
bgzf.c:1289:55: warning: comparison of integers of different signs: 'ssize_t' (aka 'long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (hwrite(fp->fp, j->comp_data, j->comp_len) != j->comp_len)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~
bgzf.c:1630:42: warning: unused parameter 'n_sub_blks' [-Wunused-parameter]
int bgzf_mt(BGZF *fp, int n_threads, int n_sub_blks)
                                         ^
bgzf.c:1870:41: warning: comparison of integers of different signs: 'unsigned long long' and 'int' [-Wsign-compare]
        ublock_size = current_block + 1 < fp->idx->moffs ? fp->idx->offs[current_block+1].uaddr-fp->idx->offs[current_block].uaddr : BGZF_MAX_BLOCK_SIZE;
                      ~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~
bgzf.c:1878:30: warning: comparison of integers of different signs: 'int' and 'uint64_t' (aka 'unsigned long long') [-Wsign-compare]
        if (fp->block_offset == ublock_size) {
            ~~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~
bgzf.c:2391:22: warning: comparison of integers of different signs: 'off_t' (aka 'long long') and 'uint64_t' (aka 'unsigned long long') [-Wsign-compare]
        if ( uoffset < fp->idx->offs[i].uaddr ) ihi = i - 1;
             ~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~
bgzf.c:2392:27: warning: comparison of integers of different signs: 'off_t' (aka 'long long') and 'uint64_t' (aka 'unsigned long long') [-Wsign-compare]
        else if ( uoffset >= fp->idx->offs[i].uaddr ) ilo = i + 1;
                  ~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~~~~
10 warnings generated.
In file included from faidx.c:41:
./htslib/hfile.h:271:35: warning: comparison of integers of different signs: 'long' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (fp->limit - fp->begin < nbytes){
            ~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~
faidx.c:727:15: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'long long' [-Wsign-compare]
    while ( l < end - beg && (c=bgzf_getc(fai->bgzf))>=0 )
            ~ ^ ~~~~~~~~~
faidx.c:763:13: warning: comparison of integers of different signs: 'hts_pos_t' (aka 'long long') and 'uint64_t' (aka 'unsigned long long') [-Wsign-compare]
    if (beg >= val->len) beg = val->len;
        ~~~ ^  ~~~~~~~~
faidx.c:764:13: warning: comparison of integers of different signs: 'hts_pos_t' (aka 'long long') and 'uint64_t' (aka 'unsigned long long') [-Wsign-compare]
    if (end >= val->len) end = val->len;
        ~~~ ^  ~~~~~~~~
faidx.c:855:22: warning: comparison of integers of different signs: 'uint64_t' (aka 'unsigned long long') and 'hts_pos_t' (aka 'long long') [-Wsign-compare]
    else if(val->len <= *p_beg_i)
            ~~~~~~~~ ^  ~~~~~~~~
faidx.c:860:22: warning: comparison of integers of different signs: 'uint64_t' (aka 'unsigned long long') and 'hts_pos_t' (aka 'long long') [-Wsign-compare]
    else if(val->len <= *p_end_i)
            ~~~~~~~~ ^  ~~~~~~~~
6 warnings generated.
header.c:59:19: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
    for (j = 0; j < len && line[j] != '\n'; j++)
                ~ ^ ~~~
1 warning generated.
In file included from hfile.c:44:
./htslib/hfile.h:271:35: warning: comparison of integers of different signs: 'long' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (fp->limit - fp->begin < nbytes){
            ~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~
hfile.c:211:16: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'ptrdiff_t' (aka 'long') [-Wsign-compare]
    if (bufsiz < curr_used)
        ~~~~~~ ^ ~~~~~~~~~
hfile.c:452:35: warning: comparison of integers of different signs: 'off_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (offset > 0 || -offset > length) {
                          ~~~~~~~ ^ ~~~~~~
hfile.c:570:28: warning: unused parameter 'fpv' [-Wunused-parameter]
static int fd_flush(hFILE *fpv)
                           ^
hfile.c:760:30: warning: unused parameter 'fpv' [-Wunused-parameter]
static off_t mem_seek(hFILE *fpv, off_t offset, int whence)
                             ^
hfile.c:760:41: warning: unused parameter 'offset' [-Wunused-parameter]
static off_t mem_seek(hFILE *fpv, off_t offset, int whence)
                                        ^
hfile.c:760:53: warning: unused parameter 'whence' [-Wunused-parameter]
static off_t mem_seek(hFILE *fpv, off_t offset, int whence)
                                                    ^
hfile.c:766:29: warning: unused parameter 'fpv' [-Wunused-parameter]
static int mem_close(hFILE *fpv)
                            ^
hfile.c:828:38: warning: unused parameter 'filename' [-Wunused-parameter]
static hFILE *hopenv_mem(const char *filename, const char *mode, va_list args)
                                     ^
hfile.c:946:51: warning: unused parameter 'scheme' [-Wunused-parameter]
static int try_exe_add_scheme_handler(const char *scheme,
                                                  ^
hfile.c:947:74: warning: unused parameter 'handler' [-Wunused-parameter]
                                      const struct hFILE_scheme_handler *handler)
                                                                         ^
hfile.c:998:64: warning: missing field 'vopen' initializer [-Wmissing-field-initializers]
        data = { hopen_mem, hfile_always_local, "built-in", 80 },
                                                               ^
hfile.c:999:71: warning: missing field 'vopen' initializer [-Wmissing-field-initializers]
        file = { hopen_fd_fileuri, hfile_always_local, "built-in", 80 },
                                                                      ^
hfile.c:1000:74: warning: missing field 'vopen' initializer [-Wmissing-field-initializers]
        preload = { hopen_preload, is_preload_url_remote, "built-in", 80 };
                                                                         ^
hfile.c:1061:67: warning: missing field 'vopen' initializer [-Wmissing-field-initializers]
        { hopen_unknown_scheme, hfile_always_local, "built-in", 0 };
                                                                  ^
hfile.c:1066:19: warning: comparison of integers of different signs: 'int' and 'unsigned long' [-Wsign-compare]
    for (i = 0; i < sizeof scheme; i++)
                ~ ^ ~~~~~~~~~~~~~
hfile.c:1073:21: warning: comparison of integers of different signs: 'int' and 'unsigned long' [-Wsign-compare]
    if (i <= 1 || i >= sizeof scheme) return NULL;
                  ~ ^  ~~~~~~~~~~~~~
hfile.c:1107:37: warning: unused parameter 'fname' [-Wunused-parameter]
int hfile_always_local (const char *fname) { return 0; }
                                    ^
hfile.c:1110:37: warning: unused parameter 'fname' [-Wunused-parameter]
int hfile_always_remote(const char *fname) { return 1; }
                                    ^
19 warnings generated.
In file included from hfile_net.c:30:
In file included from ./hfile_internal.h:31:
./htslib/hfile.h:271:35: warning: comparison of integers of different signs: 'long' and 'size_t' (aka 'unsigned long') [-Wsign-compare]
        if (fp->limit - fp->begin < nbytes){
            ~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~
hfile_net.c:106:57: warning: missing field 'vopen' initializer [-Wmissing-field-initializers]
        { hopen_net, hfile_always_remote, "knetfile", 0 };
                                                        ^
2 warnings generated.
In file included from hts.c:44:
In file included from ./cram/cram.h:46:
./header.h:227:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_fill_hrecs(sam_hdr_t *bh);
                       ^
./header.h:236:21: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_rebuild(sam_hdr_t *bh);
                    ^
./header.h:244:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_new(void);
^
./header.h:250:1: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
^
./header.h:250:28: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrecs_t *sam_hrecs_dup(sam_hrecs_t *hrecs);
                           ^
./header.h:259:34: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                 ^
./header.h:259:55: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hdr_update_target_arrays(sam_hdr_t *bh, const sam_hrecs_t *hrecs,
                                                      ^
./header.h:268:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_rebuild_text(const sam_hrecs_t *hrecs, kstring_t *ks);
                                 ^
./header.h:276:21: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
void sam_hrecs_free(sam_hrecs_t *hrecs);
                    ^
./header.h:285:41: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_type_t *sam_hrecs_find_type_id(sam_hrecs_t *hrecs, const char *type,
                                        ^
./header.h:292:26: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
int sam_hrecs_remove_key(sam_hrecs_t *hrecs,
                         ^
./header.h:302:34: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
sam_hrec_rg_t *sam_hrecs_find_rg(sam_hrecs_t *hrecs, const char *rg);
                                 ^
./header.h:305:42: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_sort_order sam_hrecs_sort_order(sam_hrecs_t *hrecs);
                                         ^
./header.h:308:44: error: must use 'struct' tag to refer to type 'sam_hrecs_t'
enum sam_group_order sam_hrecs_group_order(sam_hrecs_t *hrecs);
                                           ^
In file included from hts.c:44:
In file included from ./cram/cram.h:47:
./cram/cram_structs.h:683:5: error: must use 'struct' tag to refer to type 'sam_hdr_t'
    sam_hdr_t     *header;
    ^
    struct 
In file included from hts.c:44:
In file included from ./cram/cram.h:48:
./cram/cram_io.h:671:24: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int refs2id(refs_t *r, sam_hdr_t *hdr);
                       ^
                       struct 
./cram/cram_io.h:828:1: error: must use 'struct' tag to refer to type 'sam_hdr_t'
sam_hdr_t *cram_read_SAM_hdr(cram_fd *fd);
^
struct 
./cram/cram_io.h:836:37: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_write_SAM_hdr(cram_fd *fd, sam_hdr_t *hdr);
                                    ^
                                    struct 
./cram/cram_io.h:929:41: error: must use 'struct' tag to refer to type 'sam_hdr_t'
int cram_set_header2(cram_fd *fd, const sam_hdr_t *hdr);
                                        ^
                                        struct 
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [hts.o] Error 1
thread 'main' panicked at 'failed to build htslib', hts-sys/build.rs:117:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
brainstorm commented 4 years ago

Can you rm -rf your cloned rust-htslib (including target) and also docker rmi all docker images with names like brainstorm/cross-x86_64-unknown-linux-gnu you might have? Let's start from a clean slate ;)

mxygc commented 4 years ago

Sure, trying. Do you mean docker rmi brainstorm/cross-x86_64-unknown-linux-gnu? Does that target MAC?

brainstorm commented 4 years ago

Yes, that should work. That docker container can be run on OSX/Windows/Linux, it does not "target" any particular OS.

mxygc commented 4 years ago

Sorry, I mean which OS the resultant binary targets? brainstorm/cross-x86_64-unknown-linux-gnu seems to target linux, not MAC.

brainstorm commented 4 years ago

Aha sorry, now I get what what you mean.

Indeed you should use the "OSX" section as detailed in the README with the Filosotile musl compiler, which I haven't thoroughly tested myself since I'm using a MacBook Air as a "client" which takes way too long to compile for me.

Perhaps you might be interested in testing/coordinating with PR https://github.com/rust-bio/rust-htslib/pull/187, since it's adding support for sans-docker compilation and local install, AFAICT?

mxygc commented 4 years ago

Good point, will try.

brainstorm commented 4 years ago

I believe Patrick did compile rust-htslib locally on https://github.com/rust-bio/rust-htslib/pull/189 (otherwise testing the pre-generated bindgen for https://github.com/rust-bio/rust-htslib/pull/189/commits/da5f6e5d52ce941f7d0a924a1093cba47c412b0e and his local testing would not have worked?).

mxygc commented 4 years ago

@brainstorm rust-htslib 61b0644 now complies on OSX, thanks.