As of 1.37.0, `dylib` shared libraries no longer support interpositioning of functions defined in C

[solb]

I encountered a stable-to-stable linking regression while working with a project that compiles to an x86-64 ELF shared library for Linux. The library exports a Rust interface, but also includes some wrappers of libc functions, written in C, that need to shadow the system implementations via interpositioning. I perform the final linking step using rustc, which works correctly under rustc 1.36.0 but subtly fails under 1.37.0: the libc wrapper functions are not exported as dynamic symbols, causing the program to behave differently at runtime. The offending patchset appears to be https://github.com/rust-lang/rust/pull/59752, and I've managed to construct a minimal example to illustrate the problem...

Minimal example

The library consists of two files:

• interpose.rs defines a single-function Rust API:

  
  #![crate_type = "dylib"]

pub fn relax() { println!("Relax said the night guard"); }

* `exit.c` defines an implementation of `exit()` that should shadow the one in libc:

include

include

void exit(int ign) { (void) ign; puts("We are programmed to receive"); }

The following short program, `hotel_california.rs`, will be used to test its behavior:

extern crate interpose;

use interpose::relax; use std::os::raw::c_int;

extern { // NB: Deliberately returns () instead of ! for the purpose of this example. fn exit(_: c_int); }

fn main() { relax(); unsafe { exit(1); } println!("You can check out any time you like but you can never leave"); }

# Expected behavior (past stable releases)
This is how the program used to behave when built with a stable compiler:

$ rustc --version rustc 1.36.0 (a53f9df32 2019-07-03) $ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs $ rustc -L. -Crpath hotel_california.rs $ ./hotel_california Relax said the night guard We are programmed to receive You can check out any time you like but you can never leave $ echo $? 0

Notice that the call to `exit()` gets intercepted and does not, in fact, exit the program.

# Broken behavior (as of 1.37.0 stable)
Newer versions of the compiler result in different program output:

$ rustc --version rustc 1.37.0 $ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs $ rustc -L. -Crpath hotel_california.rs $ ./hotel_california Relax said the night guard $ echo $? 1

# Discussion: symbol table entries
The problem is evident upon examining the static and dynamic symbol tables of the `libinterpose.so` file.  When built with rustc 1.36.0, we see that `exit` is exported in the dynamic symbol table (indicated by the `D`):

$ objdump -tT libinterpose.so | grep exit$ 000000000000118c g F .text 000000000000001a exit 000000000000118c g DF .text 000000000000001a exit

In contrast, the output from rustc 1.37.0 doesn't list `exit` in the dynamic symbol table because the static symbol table lists it as a local symbol (`l`) rather than a global one (`g`):

$ objdump -tT libinterpose.so | grep exit$ 000000000000118c l F .text 000000000000001a exit

# Discussion: linker invocation
I was curious to see how rustc was invoking cc to link the program, so I traced the command-line arguments by substituting the fake linker `false`.  Here's with rustc 1.36.0:

$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs error: linking with false failed: exit code: 1 | = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "in terpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lst d-9895e8982b0a79e7" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/user/1000/rustchJWjrY/libcompiler_builtins-38e90baf978bc428.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "exit.o" = note:

error: aborting due to previous error

And with rustc 1.37.0:

$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs error: linking with false failed: exit code: 1 | = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "-Wl,--version-script=/tmp/ user/1000/rustc7Re7af/list" "interpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/usr/lib/x86_64-linux-gnu" "-lstd-6c8733432f42c6a2" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/user/1000/rustc7Re7af/libcompiler_builtins-67541964815c9eb5.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "-Wl,-soname=libinterpose.so" "exit.o" = note:

error: aborting due to previous error

Notice the newly-added `-Wl,--version-script` flag, which has no knowledge of the symbols from the `exit.o` object file.

# Discussion: static library instead of bare object file
One might be tempted to work around the problem by telling rustc about the object file so it can keep the symbols it defines global.  I tried this on rustc 1.36.0:

$ ar rs libexit.a exit.o ar: creating libexit.a $ rustc -Cprefer-dynamic -L. interpose.rs -lexit $ rustc -L. -Crpath hotel_california.rs

This has a very surprising result: the exit symbol is not present at all in `libinterpose.so`, but it does exist somewhere (the LLVM bitcode for monomorphization, maybe?) that allows the compiler to statically link it into the **executable**:

$ objdump -tT libinterpose.so | grep exit$ $ objdump -tT hotel_california | grep exit$ 0000000000001337 g F .text 000000000000001a exit 0000000000001337 g DF .text 000000000000001a Base exit

This is no good either because it leads to subtly different interposition behavior.  For example:
* Before, `exit()` could be further shadowed by libraries loaded via the `LD_PRELOAD` environment variable.  Building it directly into the executable breaks this.
* The `cc` and `ld` apply very different optimizations to `exit()` because it is now part of a PIE instead of a PIC object; depending on how wrapping is implemented, this can break it and even result in infinite recursion.
* If a C program links against `libinterpose.so`, it will no longer get the interposed version of `exit()`.  This is a very real situation for my project, because it also exports a C API via Rust's FFI.

# Possible mitigation: expose a `-Climit_rdylib_exports` command-line switch
The simplest way to allow users to work around this would be to allow invokers of rustc to opt out of the change introduced by https://github.com/rust-lang/rust/pull/59752.  However, the change is likely to have broken other use cases as well, so perhaps it needs to be revisited in more detail.

# See also
The same changeset seems to be causing problems with inline functions, as observed at https://github.com/rust-lang/rust/issues/65610.

[nagisa]

Related https://github.com/rust-lang/rust/issues/65610

[Zoxc]

This can be worked around with a #[link(name = "foo", type = "static")] attribute in interpose.rs on an public extern block declaring exit so rustc knows that needs to be exported.

[itamarst]

I hit this issue too, and the proposed workaround is not working for me, at least.

[solb]

I agree that the proposed workaround does not work. I'm still stuck on Rust 1.36 because of this change, but I've prototyped the following workaround to remove the --version-script linker flag:

1. Paste the following wrapper script into a file (named cc, for instance):

   
   #!/bin/sh

args="" for arg in "$@" do case "$arg" in -Wl,--version-script) ;; ) args="$args '$arg'" esac done

arg0="basename "$0"" eval exec "'$arg0'"$args

2. Make the script executable: `$ chmod +x cc`

3. When compiling the *library*, tell rustc to use this wrapper script instead of the system linker directly, e.g.: `$ rustc -Clinker=./cc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs`

(If using cargo, you can accomplish this as a one-off by doing `$ cargo rustc -- -Clinker=./cc` instead of `$ cargo build`, or permanently by modifying the build flags via a config file at `.cargo/config`.)

Sorry for the delay in noticing you needed this, @itamarst!

[o0Ignition0o]

@rustbot modify labels to +I-prioritize

[spastorino]

Assigning P-medium as discussed as part of the Prioritization Working Group process and removing I-prioritize.