Question/Issue when static linking in x86_64 Linux

[guenhter]

Hi,

I'm pretty certain that this is not a direct issue of this library but maybe you still can help me out and in the best case we could adapt the README to make static linking easier.

I have this main:

fn main() {
    let version = version();
    println!("{:?}", version);
}

and a lib.rs which is just the copy of the testcrate of this library.

my build.rs just is:

fn main() {
    println!("cargo:rerun-if-changed=build.rs");
    println!("cargo:rerun-if-env-changed=PROFILE");

    zeromq_src::Build::new().with_libsodium(None).build();
}

but when I run

RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-gnu

this error results:

warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ip.cpp: In function ‘void zmq::set_socket_priority(zmq::fd_t, int)’:
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ip.cpp:233:37: warning: unused parameter ‘s_’ [-Wunused-parameter]
warning:   233 | void zmq::set_socket_priority (fd_t s_, int priority_)
warning:       |                                ~~~~~^~
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ip.cpp:233:45: warning: unused parameter ‘priority_’ [-Wunused-parameter]
warning:   233 | void zmq::set_socket_priority (fd_t s_, int priority_)
warning:       |                                         ~~~~^~~~~~~~~
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/session_base.cpp: In member function ‘void zmq::session_base_t::engine_error(bool, zmq::i_engine::error_reason_t)’:
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/session_base.cpp:453:46: warning: unused parameter ‘handshaked_’ [-Wunused-parameter]
warning:   453 | void zmq::session_base_t::engine_error (bool handshaked_,
warning:       |                                         ~~~~~^~~~~~~~~~~
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/session_base.cpp:481:13: warning: this statement may fall through [-Wimplicit-fallthrough=]
warning:   481 |             if (_active) {
warning:       |             ^~
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/session_base.cpp:486:9: note: here
warning:   486 |         case i_engine::protocol_error:
warning:       |         ^~~~
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/zmtp_engine.cpp: In member function ‘bool zmq::zmtp_engine_t::handshake_v3_x(bool)’:
warning: /home/guenthgr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/zmtp_engine.cpp:345:53: warning: unused parameter ‘downgrade_sub_’ [-Wunused-parameter]
warning:   345 | bool zmq::zmtp_engine_t::handshake_v3_x (const bool downgrade_sub_)
warning:       |                                          ~~~~~~~~~~~^~~~~~~~~~~~~~
   Compiling some v0.1.0 (/home/guenthgr/dev/private/rust-noob/some)
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/guenthgr/.cargo/bin:/home/guenthgr/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/guenthgr/.npm-global/bin:/home/guenthgr/dev/private/git-extras/bin:/usr/local/go/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustc5rfwjU/symbols.o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90.some.605303045ce9a96a-cgu.0.rcgu.o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90.some.605303045ce9a96a-cgu.1.rcgu.o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90.some.605303045ce9a96a-cgu.2.rcgu.o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90.some.605303045ce9a96a-cgu.3.rcgu.o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90.1c0mur0vjo5vdtnp.rcgu.o" "-Wl,--as-needed" "-L" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/guenthgr/dev/private/rust-noob/target/release/deps" "-L" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/build/some-f56599f60914653c/out/lib" "-L" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/build/some-f56599f60914653c/out/lib" "-L" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/libsome-b3ac3b349c395dca.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0d91c78a7710ed2e.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-9bfeb974ba4dc4e7.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-7dd2d47ddb5fff81.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-18d5ce8e8a320b85.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-69d2770595dc6161.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-b156a020cc470e38.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-84091f15e468b1ee.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-fbde6acb28b510ca.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-e9de5d8ee4e7a3dd.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-9023252e4f119830.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-fcbb01769e88af40.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-f69e84994e245fea.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-27b55c02caca49ea.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-579b26075cbe9eca.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-54c30397d4b33d3b.rlib" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-lgcc_eh" "-lgcc" "-lc" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-65fb576691133eee.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-3fff6412017c0b89.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-fd3918c72578db43.rlib" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-2a597573799b576f.rlib" "-Wl,-Bdynamic" "-lstdc++" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/some-65c5ce474cb84f90" "-Wl,--gc-sections" "-static-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs"
  = note: /usr/bin/ld: /home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0d91c78a7710ed2e.rlib(std-0d91c78a7710ed2e.std.a2d7a4c8dd6a1366-cgu.0.rcgu.o): in function `std::sys::unix::os::home_dir::fallback':
          /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/sys/unix/os.rs:642: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
          /usr/bin/ld: /home/guenthgr/dev/private/rust-noob/target/x86_64-unknown-linux-gnu/release/deps/libsome-b3ac3b349c395dca.rlib(a2b07f15363ef133-ip_resolver.o): in function `zmq::ip_resolver_t::do_getaddrinfo(char const*, char const*, addrinfo const*, addrinfo**) [clone .localalias]':
          ip_resolver.cpp:(.text._ZN3zmq13ip_resolver_t14do_getaddrinfoEPKcS2_PK8addrinfoPPS3_+0x11): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
          /usr/bin/ld: /home/guenthgr/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-0d91c78a7710ed2e.rlib(std-0d91c78a7710ed2e.std.a2d7a4c8dd6a1366-cgu.0.rcgu.o): undefined reference to symbol '__tls_get_addr@@GLIBC_2.3'
          /usr/bin/ld: /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2: error adding symbols: DSO missing from command line
          collect2: error: ld returned 1 exit status

  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

Running the code without static linking works like a charm. Do you have any idea what the cause can be?

[jean-airoldie]

If I understand correctly, you want to build libzmq statically but also link statically to your GLIBC runtime, instead of the default dynamic.

I personally statically link to musl instead, although musl can have lower performance than glibc in some use cases, so I'm not too familiar with this kind of issue.

Based on the ouput you provided, the issue seems to be related to this kind of linking:

undefined reference to symbol '__tls_get_addr@@GLIBC_2.3'

After doing a little big of research, it seems that statically linking against glibc is advised against because you lose some functionality, specifically related to networking (NSS), and there also potential licensing issues. So my guess is that your error is due to glibc static linking to some functionality that can't be statically linked.

• stackoverflow link about loss of functionality
• lwn.net post about license issues

I would try static linking against musl if you can, otherwise I'm not sure I can help.

[guenhter]

Thank you so much even for checking this. Musl would be also perfeclty fine. When I run it with it, I get some error regarding musl-g++. When I create then a link for musl-g++ to point to g++ and run

RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl

I get this

running: "musl-g++" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-static" "-I" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/include" "-I" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src" "-I" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src" "-I" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/external/sha1" "-I" "/work/target/x86_64-unknown-linux-musl/release/build/zmq-sys-748afc6f53e07fb4/out" "-Wall" "-Wextra" "-DZMQ_BUILD_TESTS=OFF" "-DZMQ_USE_CV_IMPL_STL11=1" "-DZMQ_STATIC=1" "-DZMQ_USE_BUILTIN_SHA1=1" "-DZMQ_HAVE_WS=1" "-DZMQ_IOTHREAD_POLLER_USE_EPOLL=1" "-DZMQ_POLL_BASED_ON_POLL=1" "-DZMQ_HAVE_IPC=1" "-DHAVE_STRNLEN=1" "-DZMQ_HAVE_UIO=1" "-DZMQ_HAVE_STRLCPY=1" "-o" "/work/target/x86_64-unknown-linux-musl/release/build/zmq-sys-748afc6f53e07fb4/out/lib/711a6e249c8746d4-ws_listener.o" "-c" "/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ws_listener.cpp"
  cargo:warning=In file included from /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ws_engine.cpp:57:
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/compat.hpp: In instantiation of 'int strcpy_s(char (&)[size], const char*) [with long unsigned int size = 2049]':
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ws_engine.cpp:461:64:   required from here
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/compat.hpp:58:32: error: 'strlcpy' was not declared in this scope; did you mean 'strncpy'?
  cargo:warning=   58 |     const size_t res = strlcpy (dest_, src_, size);
  cargo:warning=      |                        ~~~~~~~~^~~~~~~~~~~~~~~~~~~
  cargo:warning=      |                        strncpy
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/compat.hpp: In instantiation of 'int strcpy_s(char (&)[size], const char*) [with long unsigned int size = 256]':
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/ws_engine.cpp:475:69:   required from here
  cargo:warning=/usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/zeromq-src-0.2.5+4.3.4/vendor/src/compat.hpp:58:32: error: 'strlcpy' was not declared in this scope; did you mean 'strncpy'?
  cargo:warning=   58 |     const size_t res = strlcpy (dest_, src_, size);
  cargo:warning=      |                        ~~~~~~~~^~~~~~~~~~~~~~~~~~~
  cargo:warning=      |                        strncpy
  exit status: 1

How do you link to musl?

[jean-airoldie]

Alright, so interestingly RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-gnu , which statically links against glibc runs fine on my system. So I'm guessing you are missing some required libraries on your system. My glibc version is 2.31 btw.

As for linking against musl, the command you provided would run fine:

RUSTFLAGS='-C target-feature=+crt-static' cargo build --release --target x86_64-unknown-linux-musl

The problem is that you need to have musl installed. On debian its musl, musl-dev, and musl-tools. However since debian does not provided a musl-g++ cross compiler, you would have to build it yourself, which is probably a major pain. See this link (section Building a cross compiler targeting musl libc) for more info. It would probably be simpler for you to try to make the glibc build work. I'm afraid I can't help you more than that.

[guenhter]

The interesting think is, that I do this things always in a fresh docker container of the image rust:1.71.0.

[jean-airoldie]

You could try using a rust alpine linux container since alpine uses musl natively if I recall correctly. That might work. Something like rust:1.71-alpine3.18.

[guenhter]

Running the alpine container and adding apk add g++ fixed the issue. Thx. Appreciate your excellent support.

This is absolutely good enough for my use case. Even though I'll investigate a little bit more what dependencies are missing in the normal rust image and if I find it, I'll update the findings in this issue. here.