Compatibility with LLVM6.0 on soon-to-be-release nightly

[PaulGrandperrin]

Rustc will very soon merge it's shinny new LLVM6.0 backend https://github.com/rust-lang/rust/pull/47828. It will break fuzzing with current libfuzzer because the generated coverage intrumentation will expect new symbols to be implemented by the linked fuzzer implementation.

The current libfuzzer in libfuzzer-sys is 4.0 and is missing those new symbols. Upgrading libfuzzer source code to 6.0 while disabling sanitizers works well and is compatible with compiler using LLVM4.0 (so the transition will be easy).

However, begin able to compile with sanitizer enabled will need more work: for instance, to make sanitizer=address work, we will probably need to first package asan

[nagisa]

We only support the current nightly, so this will get updated whenever a nightly with LLVM 6.0 happens.

Originally sanitizers were included only because they were required for fuzzing, however if they aren’t anymore, there’s less of the need to actually include sanitizer support with cargo fuzz. We can instead delegate users to use -Zsanitizer feature directly.

[PaulGrandperrin]

I'm reporting this in advance because the upgrade to libFuzzer6.0 with support for sanitizer is non-trivial and might take time. However, if we just drop the support for sanitizers, then the upgrade will be easy. I just wanted to help the fuzzing team be aware that a decision will soon need to be made about sanitizer support or lack thereof. If the team choose to continue to support sanitizers (which makes sense because of the unsafe{} and C libraries) then some work will need to be done before cargo-fuzz can work with rustc on LLVM6.0.

[PaulGrandperrin]

For info here is a segfault using libfuzzer6.0 and libasan6.0:

[6] INFO: Seed: 4163609845
[6] INFO: Loaded 1 modules   (77596 guards): 77596 [0x55bface8e420, 0x55bfaceda090), 
[6] INFO:      146 files found in /home/paulg/Projets/reactfs/fuzz/corpus/btree
[6] AddressSanitizer:DEADLYSIGNAL
[6] =================================================================
[6] ==25825==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000001 (pc 0x55bfac62c415 bp 0x7ffd48885eb0 sp 0x7ffd48885d60 T0)
[6] ==25825==The signal is caused by a WRITE memory access.
[6] ==25825==Hint: address points to the zero page.
[6]     #0 0x55bfac62c414  (/home/paulg/Projets/reactfs/fuzz/target/x86_64-unknown-linux-gnu/debug/btree+0x20414)
[6]     #1 0x55bfacbb681b  (/home/paulg/Projets/reactfs/fuzz/target/x86_64-unknown-linux-gnu/debug/btree+0x5aa81b)
[6]     #2 0x55bfacbb60e1  (/home/paulg/Projets/reactfs/fuzz/target/x86_64-unknown-linux-gnu/debug/btree+0x5aa0e1)
[6]     #3 0x55bfacbcd2e8  (/home/paulg/Projets/reactfs/fuzz/target/x86_64-unknown-linux-gnu/debug/btree+0x5c12e8)
[6] 
[6] AddressSanitizer can not provide additional info.
[6] SUMMARY: AddressSanitizer: SEGV (/home/paulg/Projets/reactfs/fuzz/target/x86_64-unknown-linux-gnu/debug/btree+0x20414) 
[6] ==25825==ABORTING
[6] MS: 0 ; base unit: 0000000000000000000000000000000000000000
[6] 
[6] 
[6] artifact_prefix='/home/paulg/Projets/reactfs/fuzz/artifacts/btree/'; Test unit written to /home/paulg/Projets/reactfs/fuzz/artifacts/btree/crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
[6] Base64: 
Worker 6 finished fuzzing

and the same segfault without any sanitizers:

[4] INFO: Seed: 3725420511
[4] INFO: Loaded 1 modules   (27937 guards): 27937 [0x55ecad9d2b4c, 0x55ecad9edfd0), 
[4] INFO:      146 files found in /home/paulg/Projets/reactfs/fuzz/corpus/btree
[4] ==29058== ERROR: libFuzzer: deadly signal
[4] NOTE: libFuzzer has rudimentary signal handlers.
[4]       Combine libFuzzer with AddressSanitizer or similar for better crash reports.
[4] SUMMARY: libFuzzer: deadly signal
[4] MS: 0 ; base unit: 0000000000000000000000000000000000000000
[4] 
[4] 
[4] artifact_prefix='/home/paulg/Projets/reactfs/fuzz/artifacts/btree/'; Test unit written to /home/paulg/Projets/reactfs/fuzz/artifacts/btree/crash-da39a3ee5e6b4b0d3255bfef95601890afd80709
[4] Base64: 
Worker 4 finished fuzzing

[ishitatsuyuki]

I think ThinLTO broke sanitizer and it's still a non-workaroundable problem. Was there any changes since https://github.com/rust-lang/rust/issues/45220?

[frewsxcv]

Update: The rust LLVM6.0 PR merged 11 hours ago https://github.com/rust-lang/rust/pull/47828