search

rust-bio / rust-htslib

This library provides HTSlib bindings and a high level Rust API for reading and writing BAM files.
MIT License
301 stars 77 forks source link

Cannot compile rust-htslib with x86_64-unknown-linux-musl (needed for AWS Lambda runtimes) #179

Closed brainstorm closed 4 years ago

brainstorm commented 4 years ago

Hello rust-htslib,

I'm currently using this crate to simply read a BAM header. My crate builds fine on my machine (running OSX Catalina, also tested on a regular Linux machine):

$ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 1.76s

The problem comes when I try to cross-compile it to run on AWS Lambda... it looks like the linking phase fails for the few rust-htslib functions I need to parse that BAM header:

  = note: /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h7805fb2441edac2e':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h7805fb2441edac2eE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hc749ab5efb9aea68':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17hc749ab5efb9aea68E+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::hac1f0f64c9261566':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17hac1f0f64c9261566E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h06ec3494656e04e9':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h06ec3494656e04e9E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status

Here's the full output while using serverless.com (lambda) deployment tool:

$ time npx serverless invoke local -f reads --path tests/rest/apigw_proxy_request.json
Serverless: Building native Rust reads func...
   Compiling reads v0.1.0 (/code/reads)
error: linking with `cc` failed: exit code: 1
  |
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.0.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.1.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.10.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.11.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.12.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.13.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.14.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.15.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.2.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.3.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.4.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.5.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.6.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.7.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.8.rcgu.o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.reads.2spb1sze-cgu.9.rcgu.o" "-o" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc" "/code/target/lambda/release/deps/reads-c59a3e25c753bcfc.q66hosipaldt9qk.rcgu.o" "-Wl,--gc-sections" "-pie" "-Wl,-zrelro" "-Wl,-znow" "-Wl,-O1" "-nodefaultlibs" "-L" "/code/target/lambda/release/deps" "-L" "/code/target/lambda/release/build/rust-htslib-799c0bade2369d9f/out" "-L" "/code/target/lambda/release/build/bzip2-sys-5dfe5490da2f86dc/out/lib" "-L" "/usr/lib64" "-L" "/usr/lib64" "-L" "/code/target/lambda/release/build/backtrace-sys-535b6335aaa7b508/out" "-L" "/code/target/lambda/release/build/ring-b23e7a0b1eabf848/out" "-L" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/code/target/lambda/release/deps/libenv_logger-1c4a8fe09d99d6ce.rlib" "/code/target/lambda/release/deps/libtermcolor-2e1d9c18e8508b2c.rlib" "/code/target/lambda/release/deps/libatty-bbfde0bac2726585.rlib" "/code/target/lambda/release/deps/libhumantime-f6da84e640158326.rlib" "/code/target/lambda/release/deps/libreads-bc643fc5b3e937ba.rlib" "/code/target/lambda/release/deps/librusoto_s3-401facb5057844c9.rlib" "/code/target/lambda/release/deps/librusoto_core-8701ac61d4c1d223.rlib" "/code/target/lambda/release/deps/librusoto_signature-71355f46ad7b8805.rlib" "/code/target/lambda/release/deps/libsha2-bc802680d23010d3.rlib" "/code/target/lambda/release/deps/libopaque_debug-21cdd034a7501d3b.rlib" "/code/target/lambda/release/deps/libfake_simd-3d9cfe582744e094.rlib" "/code/target/lambda/release/deps/libblock_buffer-6e610641b8cd296a.rlib" "/code/target/lambda/release/deps/libblock_padding-d009ce113c0dc642.rlib" "/code/target/lambda/release/deps/libbyte_tools-dead233fc5d5d69f.rlib" "/code/target/lambda/release/deps/libmd5-4e8e4fcd18b2caa4.rlib" "/code/target/lambda/release/deps/libhmac-a3181935e8f75ea0.rlib" "/code/target/lambda/release/deps/libcrypto_mac-a2628570ae054160.rlib" "/code/target/lambda/release/deps/libsubtle-0ec90516f40cfc7d.rlib" "/code/target/lambda/release/deps/libdigest-9108291b95da666e.rlib" "/code/target/lambda/release/deps/libgeneric_array-b732771d4701424f.rlib" "/code/target/lambda/release/deps/libtypenum-0e70489e57426234.rlib" "/code/target/lambda/release/deps/libhex-9bcef11d2a9d8fdb.rlib" "/code/target/lambda/release/deps/libxml-6ba16929dd6c929e.rlib" "/code/target/lambda/release/deps/librusoto_credential-a1acb71a2a976b7f.rlib" "/code/target/lambda/release/deps/libtokio_process-e8c69168fdd98383.rlib" "/code/target/lambda/release/deps/libtokio_signal-7642daf1d7fb4535.rlib" "/code/target/lambda/release/deps/libsignal_hook-66111ed85addbc1f.rlib" "/code/target/lambda/release/deps/libsignal_hook_registry-5872f68d13fa9064.rlib" "/code/target/lambda/release/deps/libarc_swap-6e85afef0db53f36.rlib" "/code/target/lambda/release/deps/libshlex-5756f7ae206641fc.rlib" "/code/target/lambda/release/deps/libdirs-fb4c40eec9c64f1a.rlib" "/code/target/lambda/release/deps/libbase64-171cf550deb21045.rlib" "/code/target/lambda/release/deps/libhyper_rustls-8296aad5217adcdf.rlib" "/code/target/lambda/release/deps/libwebpki_roots-8f2f4be287387b9d.rlib" "/code/target/lambda/release/deps/libtokio_rustls-55f078686cd20e1d.rlib" "/code/target/lambda/release/deps/librustls-49f2857ea172cf4c.rlib" "/code/target/lambda/release/deps/libwebpki-59e0ffd29d8392ec.rlib" "/code/target/lambda/release/deps/libct_logs-033a8f1d3fd33e7e.rlib" "/code/target/lambda/release/deps/libsct-39fa75e23df9971e.rlib" "/code/target/lambda/release/deps/libring-80e7720f9c681234.rlib" "/code/target/lambda/release/deps/libspin-d84e03c9d5f99498.rlib" "/code/target/lambda/release/deps/libuntrusted-3e073aea3169f18f.rlib" "/code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib" "/code/target/lambda/release/deps/libitertools-b4d00f4afdf9482c.rlib" "/code/target/lambda/release/deps/libieee754-31e358538504232c.rlib" "/code/target/lambda/release/deps/liburl-bf7ca148d5801d2d.rlib" "/code/target/lambda/release/deps/libpercent_encoding-d985814ec59e21ba.rlib" "/code/target/lambda/release/deps/libidna-8324e99afecc7625.rlib" "/code/target/lambda/release/deps/libbio_types-39802ccaf566d40a.rlib" "/code/target/lambda/release/deps/libquick_error-742a8a3c66b09d99.rlib" "/code/target/lambda/release/deps/libregex-ad20702983a00147.rlib" "/code/target/lambda/release/deps/libthread_local-fbdfc22547376d59.rlib" "/code/target/lambda/release/deps/libregex_syntax-bb131d70322970c6.rlib" "/code/target/lambda/release/deps/libaho_corasick-9242d4850ca1a516.rlib" "/code/target/lambda/release/deps/liblinear_map-1f78a0ead911ef1d.rlib" "/code/target/lambda/release/deps/libsnafu-e20f471e32757143.rlib" "/code/target/lambda/release/deps/libdoc_comment-c167071d06a64bad.rlib" "/code/target/lambda/release/deps/liblzma_sys-de8d3acab2d421ca.rlib" "/code/target/lambda/release/deps/libbzip2_sys-72db201261a30650.rlib" "/code/target/lambda/release/deps/liblibz_sys-f94e16d72d2a2728.rlib" "/code/target/lambda/release/deps/libnewtype_derive-3845a05c4ec64475.rlib" "/code/target/lambda/release/deps/libcustom_derive-ef5f5fe80dbc7d21.rlib" "/code/target/lambda/release/deps/libbio_index_formats-9a0b0efdc5052ebe.rlib" "/code/target/lambda/release/deps/libnom-9a57c05958f39efc.rlib" "/code/target/lambda/release/deps/liblexical_core-57660f3fd3f13ce1.rlib" "/code/target/lambda/release/deps/libstatic_assertions-956021c138bce1d0.rlib" "/code/target/lambda/release/deps/libmemchr-0e058df740fee616.rlib" "/code/target/lambda/release/deps/liblambda_http-b53f2ef9ecee8097.rlib" "/code/target/lambda/release/deps/libserde_urlencoded-00811dfcccc77fb5.rlib" "/code/target/lambda/release/deps/liburl-415f1c5bd79fc8b4.rlib" "/code/target/lambda/release/deps/libpercent_encoding-f6efc28bdf3e3b8b.rlib" "/code/target/lambda/release/deps/libidna-67d71c0abc3a4292.rlib" "/code/target/lambda/release/deps/libunicode_normalization-0b43df8177c5285a.rlib" "/code/target/lambda/release/deps/libunicode_bidi-127b6dea613aa070.rlib" "/code/target/lambda/release/deps/libmatches-656cd79ac1f4dbf3.rlib" "/code/target/lambda/release/deps/libdtoa-472f5f3f00a7735d.rlib" "/code/target/lambda/release/deps/libbase64-2ed305ffdd3ae9ce.rlib" "/code/target/lambda/release/deps/liblambda_runtime-794a2e51151d5f82.rlib" "/code/target/lambda/release/deps/liblambda_runtime_core-6eefdfd89ee7e561.rlib" "/code/target/lambda/release/deps/liblambda_runtime_client-87bf1eb260a4eaf3.rlib" "/code/target/lambda/release/deps/liblambda_runtime_errors-d64afb19e0dbdd5c.rlib" "/code/target/lambda/release/deps/libserde_json-c8a351a1c05e9114.rlib" "/code/target/lambda/release/deps/libryu-4ee08cd2532a545c.rlib" "/code/target/lambda/release/deps/libhyper-cf93579cb192ec79.rlib" "/code/target/lambda/release/deps/libwant-15b092e4a3d6a406.rlib" "/code/target/lambda/release/deps/libtry_lock-c3ab9ae9b8d10427.rlib" "/code/target/lambda/release/deps/libtokio-13b44038747fd62f.rlib" "/code/target/lambda/release/deps/libtokio_uds-1494cc16fafcd85f.rlib" "/code/target/lambda/release/deps/libmio_uds-a5228904c2869e8b.rlib" "/code/target/lambda/release/deps/libtokio_udp-561e74841b75bbac.rlib" "/code/target/lambda/release/deps/libtokio_timer-5b97631b8eb96162.rlib" "/code/target/lambda/release/deps/libtokio_tcp-995e65fd9a2000ff.rlib" "/code/target/lambda/release/deps/libtokio_reactor-339239976facad32.rlib" "/code/target/lambda/release/deps/libtokio_sync-f485e5f095e03bec.rlib" "/code/target/lambda/release/deps/libparking_lot-159d76519bc01e87.rlib" "/code/target/lambda/release/deps/libparking_lot_core-467e99363d50e2f7.rlib" "/code/target/lambda/release/deps/libsmallvec-5b05b57becb06706.rlib" "/code/target/lambda/release/deps/liblock_api-606b6eac909b2c11.rlib" "/code/target/lambda/release/deps/libtokio_fs-6a3a93cb8fdc7c9b.rlib" "/code/target/lambda/release/deps/libtokio_threadpool-88f4ea135c22fcb3.rlib" "/code/target/lambda/release/deps/libcrossbeam_queue-55b75d73ff737e2c.rlib" "/code/target/lambda/release/deps/libcrossbeam_deque-112850f4e0e976bf.rlib" "/code/target/lambda/release/deps/libcrossbeam_epoch-fddda5ae3a49d8dd.rlib" "/code/target/lambda/release/deps/libscopeguard-fbfb32c1cb4c3e17.rlib" "/code/target/lambda/release/deps/libmemoffset-fb5d6f6521a0c490.rlib" "/code/target/lambda/release/deps/libarrayvec-a45440c67092cb7a.rlib" "/code/target/lambda/release/deps/libnodrop-bdc84fbe608ce13e.rlib" "/code/target/lambda/release/deps/libtokio_current_thread-fe92cb9121a42232.rlib" "/code/target/lambda/release/deps/libtokio_executor-fcbd3244dcc6fd85.rlib" "/code/target/lambda/release/deps/libcrossbeam_utils-a3643d7f61cb12cf.rlib" "/code/target/lambda/release/deps/liblazy_static-9927bdb5a51d8866.rlib" "/code/target/lambda/release/deps/libtokio_codec-17dc43c924dd6243.rlib" "/code/target/lambda/release/deps/libmio-7201b2b7387f7aa3.rlib" "/code/target/lambda/release/deps/libnet2-4644b054ec10f0f1.rlib" "/code/target/lambda/release/deps/libhttparse-b16dabd259454e4e.rlib" "/code/target/lambda/release/deps/libhttp_body-6aaf78b60f341d5d.rlib" "/code/target/lambda/release/deps/libtokio_buf-9d4b34bec93a1efa.rlib" "/code/target/lambda/release/deps/libh2-906f495e546f7d18.rlib" "/code/target/lambda/release/deps/libindexmap-b77f8a21b11eb0e9.rlib" "/code/target/lambda/release/deps/libstring-1ca39da16cef2a4d.rlib" "/code/target/lambda/release/deps/libslab-aca7457c7f2adc8c.rlib" "/code/target/lambda/release/deps/libtokio_io-a43ce87c775afd43.rlib" "/code/target/lambda/release/deps/liblog-5f075c2f94d34499.rlib" "/code/target/lambda/release/deps/libfutures_cpupool-0dce3024d5bb313f.rlib" "/code/target/lambda/release/deps/libnum_cpus-b351c5920ba4ec23.rlib" "/code/target/lambda/release/deps/libfutures-d6d90b26a1db9345.rlib" "/code/target/lambda/release/deps/libchrono-da675f9399653e59.rlib" "/code/target/lambda/release/deps/libserde-76418f02191bd7eb.rlib" "/code/target/lambda/release/deps/libnum_integer-daa3b551460da5a5.rlib" "/code/target/lambda/release/deps/libnum_traits-a9b752e1c8cb274d.rlib" "/code/target/lambda/release/deps/libtime-5c0346007023d7b7.rlib" "/code/target/lambda/release/deps/libfailure-5f92cd4325d7f6ec.rlib" "/code/target/lambda/release/deps/libbacktrace-45eba65ba84e9c3f.rlib" "/code/target/lambda/release/deps/libbacktrace_sys-22c3e4d94d53b8b3.rlib" "/code/target/lambda/release/deps/libcfg_if-f51889ee12a5a31e.rlib" "/code/target/lambda/release/deps/librustc_demangle-2518b9cc69d3ccce.rlib" "/code/target/lambda/release/deps/libhttp-d0f67467b0300bf9.rlib" "/code/target/lambda/release/deps/libitoa-8b90c0416aaa1224.rlib" "/code/target/lambda/release/deps/libfnv-e24d8f06a947b9e8.rlib" "/code/target/lambda/release/deps/libbytes-9b9a39b2b28a2835.rlib" "/code/target/lambda/release/deps/libeither-49105ea47ad0d133.rlib" "/code/target/lambda/release/deps/libiovec-bb59257854a157b9.rlib" "/code/target/lambda/release/deps/liblibc-cc7ff7776d163983.rlib" "/code/target/lambda/release/deps/libbyteorder-8dbacdfd1cfce70a.rlib" "-Wl,--start-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-fae576517123aa4e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-a72070139220275e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-093434daf7d99801.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-24daf38551b7a03b.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace-36d70d9746402ce9.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libbacktrace_sys-7acfc843240167a8.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-eb2e0f5fe057b8b3.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-75e9ddd83715a368.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-af51e7c6fd7d1248.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-27f2a77b2995d98c.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-ad10152c26711a1e.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-291bd2456cb6c9fe.rlib" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fc6e9071307a3016.rlib" "-Wl,--end-group" "/root/.rustup/toolchains/1.39.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-ebe4001ded7f33e7.rlib" "-Wl,-Bdynamic" "-llzma" "-lz" "-lutil" "-lutil" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil"
  = note: /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::Reader::new::h7805fb2441edac2e':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam6Reader3new17h7805fb2441edac2eE+0x4c): undefined reference to `sam_hdr_read'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$::drop::hc749ab5efb9aea68':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN66_$LT$rust_htslib..bam..Reader$u20$as$u20$core..ops..drop..Drop$GT$4drop17hc749ab5efb9aea68E+0x5): undefined reference to `hts_close'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `rust_htslib::bam::hts_open::hac1f0f64c9261566':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN11rust_htslib3bam8hts_open17hac1f0f64c9261566E+0xc3): undefined reference to `hts_open'
          /code/target/lambda/release/deps/librust_htslib-fc86895fccfb6420.rlib(rust_htslib-fc86895fccfb6420.rust_htslib.25lsx2fx-cgu.11.rcgu.o): In function `_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$::drop::h06ec3494656e04e9':
          rust_htslib.25lsx2fx-cgu.11:(.text._ZN70_$LT$rust_htslib..bam..HeaderView$u20$as$u20$core..ops..drop..Drop$GT$4drop17h06ec3494656e04e9E+0xb): undefined reference to `bam_hdr_destroy'
          collect2: error: ld returned 1 exit status

error: aborting due to previous error

error: could not compile `reads`.

To learn more, run the command again with --verbose.
Serverless: Dockerized Rust build encountered an error: undefined 1.

  Error --------------------------------------------------

  Error:
      at /Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless-rust/index.js:129:15
      at Array.forEach (<anonymous>)
      at RustPlugin.build (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless-rust/index.js:110:22)
      at /Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/serverless/lib/classes/PluginManager.js:490:55
      at tryCatcher (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/util.js:16:23)
      at Object.gotValue (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/reduce.js:168:18)
      at Object.gotAccum (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/reduce.js:155:25)
      at Object.tryCatcher (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromise0 (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:649:10)
      at Promise._settlePromises (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/promise.js:729:18)
      at _drainQueueStep (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:93:12)
      at _drainQueue (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (/Users/romanvg/dev/umccr/htsget/htsget-aws/node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:439:21)
      at process.topLevelDomainCallback (domain.js:130:23)

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Issues:        forum.serverless.com

  Your Environment Information ---------------------------
     Operating System:          darwin
     Node Version:              12.14.1
     Framework Version:         1.61.3
     Plugin Version:            3.2.7
     SDK Version:               2.3.0
     Components Core Version:   1.1.2
     Components CLI Version:    1.4.0

real    0m45.065s
user    0m3.069s
sys 0m1.341s

I don't have much experience with the rust-htslib approach to cbindgen & co, but if you could provide some pointers on what you think could be wrong here, I'd be happy to investigate further.

In order to reproduce this issue, assuming you are on Linux:

$ git clone https://github.com/brainstorm/htsget-aws && cd htsget-aws
$ brew install filosottile/musl-cross/musl-cross (if using OSX and brew)
$ rustup target add x86_64-unknown-linux-musl
$ npm i
$ time npx serverless invoke local -f reads --path tests/rest/apigw_proxy_request.json

If something fails on your machine, please take a look at this blogpost which contains additional directions should you encounter some difficulty with the cross-compiling toolchain.

I would like to fix this before jumping into another rust implementation to just read the first BGZF block of a BAM file to get the header :-S

luizirber commented 4 years ago

I think there are issues even before you get to rust-htslib:

brainstorm commented 4 years ago

True to all you said. I'll be digging on the serverless-rust and docker container innards soon, closing as INVALID ;)

brainstorm commented 4 years ago

Back to this after a while... that was a not-so-INVALID issue, the path for MUSL + C bindings in Rust is pretty tortuous these days, see:

https://users.rust-lang.org/t/solved-musl-cross-compilation-how-to-reuse-curl-sys-libz-sys-bzip2-sys-lzma-sys-and-other-dependency-includes-under-target/42817/8

I'm planning to file an idea/pre-RFC to Rust internals, inquiring why *-sys crates objects and headers are not properly reused... in my view that'll solve a lot of problems and even get rid of the need to use Cross/Docker at all, but I might be missing several intricacies of Cargo, we'll see. Track this PR https://github.com/rust-bio/rust-htslib/pull/193 if anybody requires more gory details.

Thanks @luizirber for reviewing this back in January ;)