Fuzz mode as documented in https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md doesn't work

[arvid-norlander]

I followed https://github.com/google/fuzztest/blob/main/doc/quickstart-cmake.md exactly. When I tried with -DFUZZTEST_FUZZING_MODE=on it turns out it doesn't actually work:

Note: Google Test filter = MyTestSuite.IntegerAdditionCommutes
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from MyTestSuite
[ RUN      ] MyTestSuite.IntegerAdditionCommutes
FUZZTEST_PRNG_SEED=FEIOIwS7oGOzZwJD92Joo9IKQL1v4SFl1EB0uWiBrqU

[!] To fuzz, please build with --config=fuzztest.

/home/tmgarn/workspace/cmake_reproducer/fuzztest/./fuzztest/internal/googletest_adaptor.h:66: Failure
Expected equality of these values:
  0
  test->RunInFuzzingMode(argc_, argv_, configuration_)
    Which is: 1
Fuzzing failure.
Stack trace:
  0x559830210fb9: fuzztest::internal::GTest_TestAdaptor::TestBody()
  0x5598302a1adb: testing::internal::HandleSehExceptionsInMethodIfSupported<>()
  0x55983028633a: testing::internal::HandleExceptionsInMethodIfSupported<>()
  0x559830264783: testing::Test::Run()
  0x559830265380: testing::TestInfo::Run()
... Google Test internal frames ...

=================================================================
=== Fuzzing stats

Elapsed time: 2.231779ms
Total runs: 0
Edges covered: 0
Total edges: 0
Corpus size: 0
Max stack used: 0

==================================

Digging into the code it appears that this depends on something called "centipede" for which there is no code in the Cmake build system, only the bazel one seems to have support

[formatc2013]

I only run into this if I compile without: set(FUZZTEST_FUZZING_MODE ON)

My CMakeLists.txt

cmake_minimum_required(VERSION 3.19)
project(first_fuzz_project)

set(FUZZTEST_FUZZING_MODE ON)

set(CMAKE_CXX_STANDARD 20)

add_subdirectory(fuzztest)
set(CMAKE_THREAD_PREFER_PTHREAD TRUE)
set(THREADS_PREFER_PTHREAD_FLAG TRUE)
find_package(Threads REQUIRED)

enable_testing()

include(GoogleTest)

fuzztest_setup_fuzzing_flags()

add_executable(first_fuzz_test common.h first_fuzz_test.cpp)

target_link_libraries(first_fuzz_test PRIVATE xyz)

link_fuzztest(first_fuzz_test)
gtest_discover_tests(first_fuzz_test)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

Fails with:

[1/2] Linking CXX executable output/first_fuzz_test

==19646==The following global variable is not properly aligned.
==19646==This may happen if another global with the same name
==19646==resides in another non-instrumented module.
==19646==Or the global comes from a C file built w/o -fno-common.
==19646==In either case this is likely an ODR violation bug,
==19646==but AddressSanitizer can not provide more details.
=================================================================
==19646==ERROR: AddressSanitizer: odr-violation (0x5585d812aecf):
  [1] size=35 'typeinfo name for testing::internal::DeathTestImpl' deps/googletest-src/googletest/src/gtest-all.cc
  [2] size=35 'typeinfo name for testing::internal::DeathTestImpl' deps/googletest-src/googletest/src/gtest-all.cc
These globals were registered at these points:
...
    #2 0x7f06f57dc47d in call_init elf/./elf/dl-init.c:70:3

==4120==HINT: if you don't care about these errors you may set ASAN_OPTIONS=detect_odr_violation=0
SUMMARY: AddressSanitizer: odr-violation: global 'typeinfo name for testing::internal::DeathTestImpl' at 

export ASAN_OPTIONS=detect_odr_violation=0 seems to have no effect

This helps: add_compile_options(-mllvm -asan-use-private-alias=1)

but than I have to: export ASAN_OPTIONS=alloc_dealloc_mismatch=0

and than back at your problem

[cyclicreferences]

@arvid-norlander did you manage to get it working with cmake?

[cyclicreferences]

I could get it working in compatibility mode with libfuzzer. The errors I received was because I publicly linked GTest somewhere else and the cmake from the setup also links against it

[arvid-norlander]

I managed to get it working partially, but it was really finicky and didn't work well with the rest of our cmake build system ("backend" is a terrible name in anything that has a global namespace for names, neither fuzztest nor our code should be using it!). So I did a short test, but never anything that got incorporated into our CI.

I never got the continuous fuzzing mode working at all.

I'm more likely to look at other solutions in the future than this project. It seems to be the typical google "dump code as FOSS, but don't make it well documented or work well and ignore all outside bugs and contributions". Don't really see what google is getting out of that to be honest.