Always starting from an empty corpus

[brayniac]

This might be human error, but it appears libfuzzer is running with an empty corpus, even after it has generated content for the corpus. Passing the corpus directory in on the invocation of cargo fuzz run does not appear to correct the issue. I can try to come up with a minimum repro, but wanted to see if this is something others have seen.

[nagisa]

Seems to work fine here.

cargo fuzz run fuzzer_script_1 executes

fuzz/target/x86_64-unknown-linux-gnu/debug/fuzzer_script_1 -artifact_prefix=fuzz/artifacts/fuzzer_script_1/ fuzz/corpus/fuzzer_script_1

The last component is the path to the corpus. Putting a file with value which makes the fuzz script panic in the fuzz/corpus/fuzzer_script_1/somefile makes fuzzer find the panic immediately.

cargo fuzz run fuzzer_script_1 corp also passes corp as the corpus directory to the underlying binary correctly.

[brayniac]

So, I created a sample library for fuzzing, which serves as the minimum repro for this. My expectation is that on the first run of cargo fuzz run fuzzer_script_1, a corpus is created. I can see files in there with byte content. On a second run, I expect the message "INFO: A corpus is not provided, starting from empty corpus" to not be present.

# Some modification required if not using nightly by default
git clone https://github.com/brayniac/woolsweater && cd woolsweater && cargo fuzz run fuzzer_script_1; cargo fuzz run fuzzer_script_1

For me this produces the output:

# initial cargo build omitted
INFO: Seed: 1921789099
INFO: Loaded 0 modules (0 guards):
Loading corpus dir: /home/bmartin/woolsweater/fuzz/corpus/fuzzer_script_1
INFO: -max_len is not provided, using 64
INFO: A corpus is not provided, starting from an empty corpus
#0  READ units: 1
#1  INITED cov: 83 corp: 1/1b exec/s: 0 rss: 91Mb
#6  NEW    cov: 84 corp: 2/10b exec/s: 0 rss: 99Mb L: 9 MS: 5 CopyPart-CopyPart-EraseBytes-ChangeBinInt-CMP- DE: "\xff\xff\xff\xff\xff\xff\xff\xff"-
#205    NEW    cov: 92 corp: 3/14b exec/s: 0 rss: 99Mb L: 4 MS: 4 InsertByte-InsertByte-ShuffleBytes-InsertByte-
thread '<unnamed>' panicked at 'at the disco', src/lib.rs:3
note: Run with `RUST_BACKTRACE=1` for a backtrace.
==96260== ERROR: libFuzzer: deadly signal
    #0 0x7f7ce73145f3 in __sanitizer_print_stack_trace /checkout/src/compiler-rt/lib/asan/asan_stack.cc:38
    #1 0x7f7ce720ccb5 in fuzzer::Fuzzer::CrashCallback() /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerLoop.cpp:280
    #2 0x7f7ce720cc03 in fuzzer::Fuzzer::StaticCrashSignalCallback() /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerLoop.cpp:264
    #3 0x7f7ce72328cb in fuzzer::CrashHandler(int, siginfo_t*, void*) /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerUtilPosix.cpp:37
    #4 0x7f7ce66a236f  (/lib64/libpthread.so.0+0xf36f)
    #5 0x7f7ce60f11d6 in __GI_raise (/lib64/libc.so.6+0x351d6)
    #6 0x7f7ce60f28c7 in __GI_abort (/lib64/libc.so.6+0x368c7)
    #7 0x7f7ce7245e58 in panic_abort::__rust_start_panic::abort /checkout/src/libpanic_abort/lib.rs:61
    #8 0x7f7ce7245e58 in __rust_start_panic /checkout/src/libpanic_abort/lib.rs:56

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 1 CMP- DE: "halt"-; base unit: aab132499852a91538881f570e2d56ee42e488eb
0x68,0x61,0x6c,0x74,
halt
artifact_prefix='/home/bmartin/woolsweater/fuzz/artifacts/fuzzer_script_1/'; Test unit written to /home/bmartin/woolsweater/fuzz/artifacts/fuzzer_script_1/crash-c4bb896961f2dfd5f739d87783887e08f07bb4d6
Base64: aGFsdA==
       Fresh gcc v0.3.45
       Fresh woolsweater v0.1.0 (file:///home/bmartin/woolsweater)
       Fresh libfuzzer-sys v0.1.0 (https://github.com/rust-fuzz/libfuzzer-sys.git#36a3928e)
       Fresh woolsweater-fuzz v0.0.1 (file:///home/bmartin/woolsweater/fuzz)
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `fuzz/target/x86_64-unknown-linux-gnu/debug/fuzzer_script_1 -artifact_prefix=/home/bmartin/woolsweater/fuzz/artifacts/fuzzer_script_1/ /home/bmartin/woolsweater/fuzz/corpus/fuzzer_script_1`
INFO: Seed: 2254817014
INFO: Loaded 0 modules (0 guards):
Loading corpus dir: /home/bmartin/woolsweater/fuzz/corpus/fuzzer_script_1
INFO: -max_len is not provided, using 64
INFO: A corpus is not provided, starting from an empty corpus
#0  READ units: 1
#1  INITED cov: 83 corp: 1/1b exec/s: 0 rss: 89Mb
#9  NEW    cov: 84 corp: 2/53b exec/s: 0 rss: 97Mb L: 52 MS: 3 ChangeByte-ChangeBit-InsertRepeatedBytes-
#154    NEW    cov: 92 corp: 3/58b exec/s: 0 rss: 99Mb L: 5 MS: 3 ShuffleBytes-ShuffleBytes-CMP- DE: "\x00\x00\x00\x00"-
thread '<unnamed>' panicked at 'at the disco', src/lib.rs:3
note: Run with `RUST_BACKTRACE=1` for a backtrace.
==96565== ERROR: libFuzzer: deadly signal
    #0 0x7f2621bc45f3 in __sanitizer_print_stack_trace /checkout/src/compiler-rt/lib/asan/asan_stack.cc:38
    #1 0x7f2621abccb5 in fuzzer::Fuzzer::CrashCallback() /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerLoop.cpp:280
    #2 0x7f2621abcc03 in fuzzer::Fuzzer::StaticCrashSignalCallback() /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerLoop.cpp:264
    #3 0x7f2621ae28cb in fuzzer::CrashHandler(int, siginfo_t*, void*) /home/bmartin/.cargo/git/checkouts/libfuzzer-sys-e07fde05820d7bc6/36a3928/llvm/lib/Fuzzer/FuzzerUtilPosix.cpp:37
    #4 0x7f2620f5236f  (/lib64/libpthread.so.0+0xf36f)
    #5 0x7f26209a11d6 in __GI_raise (/lib64/libc.so.6+0x351d6)
    #6 0x7f26209a28c7 in __GI_abort (/lib64/libc.so.6+0x368c7)
    #7 0x7f2621af5e58 in panic_abort::__rust_start_panic::abort /checkout/src/libpanic_abort/lib.rs:61
    #8 0x7f2621af5e58 in __rust_start_panic /checkout/src/libpanic_abort/lib.rs:56

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 5 ChangeBit-InsertByte-ChangeByte-EraseBytes-CMP- DE: "crash"-; base unit: 70a63663c737770932d6e10a9644c519e05b083f
0x63,0x72,0x61,0x73,0x68,
crash
artifact_prefix='/home/bmartin/woolsweater/fuzz/artifacts/fuzzer_script_1/'; Test unit written to /home/bmartin/woolsweater/fuzz/artifacts/fuzzer_script_1/crash-2fc7f1452374b6e341d67717f032abbe0da0f4a6
Base64: Y3Jhc2g=

[nagisa]

Cannot reproduce. Will need more information. OS/Filesystem/Versions/Are there files in corpus directory after first run/strace (or equivalent) of the underlying executable (./target/release/fuzzer_script_1)/does running the underlying executable work/does doing this on tmpfs work/etc etc.

[brayniac]

Environment details: (vmware fusion guest)

$ cat /etc/redhat-release
CentOS Linux release 7.3.1611 (Core)
$ uname -r
3.10.0-514.6.1.el7.x86_64
$ mount | grep centos-root
/dev/mapper/centos-root on / type xfs (rw,relatime,seclabel,attr2,inode64,noquota)
$ rustc --version
rustc 1.19.0-nightly (d3abc80b3 2017-05-09)

[brayniac]

Yes, there are files with content in the fuzz/corpus/fuzzer_script_1 path. Adding additional corpus paths on cargo fuzz invocation shows the same behavior (corpus not loaded):

mkdir corpus && echo "halt" >> corpus/halt && cargo fuzz run fuzzer_script_1 corpus

[brayniac]

Does seem to work on tmpfs when I cp -rpv over to a new directory on tmpfs...

[brayniac]

xfs and ext4 both work fine for a loop device - however, still experiencing this issue in directories on the root filesystem. This is a pretty stock CentOS 7 install.

[brayniac]

This appears like it might be related to: https://bugs.llvm.org//show_bug.cgi?id=25991

[whitequark]

@brayniac would you be willing to build rustc and LLVM from scratch if I provided you with an LLVM patch?

[brayniac]

@whitequark - sure, I should be able to build and test if you have a LLVM patch for this. Let me know

[sanmai-NL]

@whitequark: you added a misspelled issue label, perhaps you want to fix it for future reference.

[frewsxcv]

@whitequark: you added a misspelled issue label, perhaps you want to fix it for future reference.

fixed! thanks for pointing it out