Probable problem with ASAN

[glandium]

I stumbled upon this crate on r/rust. I haven't tried it because I'm currently not on a machine that supports ASAN (arm64), but quickly looking at the code, combined with my experience maintaining something similar in Firefox, this crate likely fails in bad ways when building with ASAN. What happens in that case is that each item actually ends up larger than their size, to allow ASAN to do its magic. Which means a) the list is actually larger (in bytes) than expected b) there's a bunch of 0 bytes in between items. In Firefox, what was being aggregated was pointers, so the nulls between items were the same size as the items themselves, but I wouldn't be surprised if the size of the nulls doesn't match with larger items.

[danakj]

On Linux with ASAN this works ok. On MacOS ARM64 and iOS ARM64 with ASAN this panics. It works correctly on MacOs and iOS ARM64 without ASAN enabled.

thread '<unnamed>' panicked at 'duplicate #[distributed_slice] with name "GTESTS"', ./../../third_party/rust/linkme/v0_3/crate/src/distributed_slice.rs:257:13
stack backtrace:
[1]    12878 abort      RUST_BACKTRACE=full out_desktop/Release/rust_gtest_interop_unittests

I have a single static #[distributed_slice] in a single crate, but with ASAN it's being duplicated.

[danakj]

Followup that MacOS Intel has the same problem with ASAN as above, but works without ASAN.

I also tried with the used_linked feature enabled (for addressing https://github.com/dtolnay/linkme/issues/49) and this had no effect on ASAN.

[danakj]

Mac ASAN repro steps, on an M1 Mac with Xcode installed:

1. rustup toolchain install nightly-aarch64-apple-darwin # For ASAN
2. rustup default nightly # For ASAN
3. rustup target add x86_64-apple-darwin # To avoid ASAN in proc-macros, you need to cross-compile tests.
4. Create .cargo/config.toml:

   [target.x86_64-apple-darwin]
   rustflags = [
   "-Zsanitizer=address",
   "-Clinker=clang",
   "-Clink-args=-isysroot",
   "-Clink-arg=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk"
   ]

5. cargo test --target=x86_64-apple-darwin

   
   Finished test [unoptimized + debuginfo] target(s) in 0.29s
    Running unittests src/lib.rs (target/x86_64-apple-darwin/debug/deps/linkme-636e3cafcc3a381e)
   linkme-636e3cafcc3a381e(54687,0x20481c280) malloc: nano zone abandoned due to inability to reserve vm space.

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

 Running tests/compiletest.rs (target/x86_64-apple-darwin/debug/deps/compiletest-dbc8d549c6fc91c0)

compiletest-dbc8d549c6fc91c0(54691,0x201f89280) malloc: nano zone abandoned due to inability to reserve vm space.

running 1 test Checking linkme-tests v0.0.0 (/Users/danakj/s/linkme/target/tests/trybuild/linkme) Finished dev [unoptimized + debuginfo] target(s) in 0.12s

test tests/ui/bad_crate_path.rs ... ok test tests/ui/generic_fn.rs ... ok test tests/ui/mismatched_types.rs ... ok test tests/ui/mutable.rs ... ok test tests/ui/unsupported_item.rs ... ok test tests/ui/zerosized.rs ... ok

test ui ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.42s

 Running tests/custom_linkme_path.rs (target/x86_64-apple-darwin/debug/deps/custom_linkme_path-96558caee8082fe4)

custom_linkme_path-96558caee8082fe4(54705,0x202891280) malloc: nano zone abandoned due to inability to reserve vm space.

running 2 tests test declaration::test_functions ... FAILED test declaration::test_slice ... FAILED

failures:

---- declaration::test_functions stdout ---- thread 'declaration::test_functions' panicked at 'duplicate #[distributed_slice] with name "FUNCTIONS"', /Users/danakj/s/linkme/src/distributed_slice.rs:258:13 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- declaration::test_slice stdout ---- thread 'declaration::test_slice' panicked at 'duplicate #[distributed_slice] with name "SLICE"', /Users/danakj/s/linkme/src/distributed_slice.rs:258:13

failures: declaration::test_functions declaration::test_slice

test result: FAILED. 0 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

error: test failed, to rerun pass --test custom_linkme_path

If you're not on an M1 mac, I don't know how to get cargo test to work with ASAN, as it can't get applied to the proc macros and Cargo doesn't give you any other way to avoid it afaict. It's easier to reproduce then in Chromium probably (kinda ironically) by enabling the tests in //testing/rust_gtest_interop/BUILD.gn (remove the !is_apple at the top) and build the rust_gtest_interop_unittests with `is_asan=true` in the GN args. Chromium getting started docs describe how to set up a Chomium build. (This feels like a big gap in Cargo.)

[danakj]

The ASAN (x86_64) and non-ASAN (aarch64) look similar:

---- test_empty stdout ----
thread 'test_empty' panicked at 'duplicate #[distributed_slice] with name "EMPTY"', /Users/danakj/s/linkme/src/distributed_slice.rs:258:13

➜  linkme git:(master) ✗ nm target/debug/deps/distributed_slice-50981b83dc1ed600|grep EMPTY
00000001000a8460 s __ZN17distributed_slice10test_empty5EMPTY17h1dde0a73b27099a5E
00000001000b4008 s __ZN17distributed_slice10test_empty5EMPTY8DUPCHECK17h079a5cf71eed0d73E
➜  linkme git:(master) ✗ nm target/x86_64-apple-darwin/debug/deps/distributed_slice-7cbbeabb728f094e|grep EMPTY
00000001000c0700 s __ZN17distributed_slice10test_empty5EMPTY17hc83dcb2cbd73296eE
00000001000ce100 s __ZN17distributed_slice10test_empty5EMPTY8DUPCHECK17hbff3a324c1f391b2E

[danakj]

I thought about just removing the panic, but it actually does fail some tests on Apple-ASAN, though not all:

     Running tests/distributed_slice.rs (target/x86_64-apple-darwin/debug/deps/distributed_slice-7cbbeabb728f094e)
distributed_slice-7cbbeabb728f094e(63220,0x20342b280) malloc: nano zone abandoned due to inability to reserve vm space.

running 5 tests
test test_elided_lifetime ... ok
test test_empty ... ok
test test_non_copy ... ok
test test_interior_mutable ... FAILED
test test ... FAILED

failures:

---- test_interior_mutable stdout ----
thread 'test_interior_mutable' panicked at 'assertion failed: MUTABLE.len() == 1', tests/distributed_slice.rs:59:5

---- test stdout ----
thread 'test' panicked at 'assertion failed: `(left == right)`
  left: `24`,
 right: `3`', tests/distributed_slice.rs:20:5

[danakj]

Changing the tests to verify the number of items in the list, yeah they all fail. Each #[distributed_slice] tagged static variable is contributing itself to the slice multiple times.

---- test_interior_mutable stdout ----
thread 'test_interior_mutable' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `1`', tests/distributed_slice.rs:59:5

---- test stdout ----
thread 'test' panicked at 'assertion failed: `(left == right)`
  left: `24`,
 right: `3`', tests/distributed_slice.rs:20:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test_elided_lifetime stdout ----
thread 'test_elided_lifetime' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `1`', tests/distributed_slice.rs:71:5

[danakj]

Ah you don't need to cross compile, you can set the rustflags for your host in .cargo/config.toml and as long as you pass --target=<your host abi> it will use those for cargo tests but not for proc macros. So this should be reproducible on an Mac Intel machine too with the same config and cargo test command line above.