search

google / atheris

Apache License 2.0
1.38k stars 111 forks source link

Help with installation on ARM #44

Closed chrisruk closed 1 year ago

chrisruk commented 2 years ago

Hi,

I'm just wondering if anyone might be able to point me in the right direction, with installing atheris on ARM (using a raspberry pi 4, with the 32 bit OS).

I installed llvm/clang 11.

And am doing:

LIBFUZZER_LIB=/usr/lib/llvm-11/lib/libFuzzer.a CLANG_BIN=/usr/bin/clang CC=/usr/bin/clang CXX=/usr/bin/clang++ pip3 install .

But get:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/Repositories/atheris
Building wheels for collected packages: atheris
  Building wheel for atheris (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-rsosxi4i/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-rsosxi4i/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-tvtqegpo
       cwd: /tmp/pip-req-build-rsosxi4i/
  Complete output (70 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-armv7l-3.9
  copying atheris_no_libfuzzer.py -> build/lib.linux-armv7l-3.9
  creating build/lib.linux-armv7l-3.9/atheris
  copying src/coverage_test_helper.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/version_dependent.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/regex_match_generation_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/coverage_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzzed_data_provider_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_mutator_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzz_test_lib.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/hook-atheris.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/import_hook.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/function_hooks.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/__init__.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/utils.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/pyinstaller_coverage_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_mutator_and_crossover_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/instrument_bytecode.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_crossover_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  running build_ext
  Your libFuzzer version is too old, but it's possible to attempt an in-place upgrade. Trying that now.
  Your libFuzzer is up-to-date.
  creating tmp
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c /tmp/tmppjc2uk99.cpp -o tmp/tmppjc2uk99.o -std=c++14
  building 'atheris.native' extension
  creating build/temp.linux-armv7l-3.9
  creating build/temp.linux-armv7l-3.9/src
  creating build/temp.linux-armv7l-3.9/src/native
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/atheris.cc -o build/temp.linux-armv7l-3.9/src/native/atheris.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/fuzzed_data_provider.cc -o build/temp.linux-armv7l-3.9/src/native/fuzzed_data_provider.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  src/native/fuzzed_data_provider.cc:161:23: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
      for (int i = 0; i < bytes; ++i) {
                      ~ ^ ~~~~~
  src/native/fuzzed_data_provider.cc:212:23: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
      for (int i = 0; i < bytes; ++i) {
                      ~ ^ ~~~~~
  src/native/fuzzed_data_provider.cc:284:44: warning: implicit conversion from 'unsigned long long' to 'const double' changes value from 18446744073709551615 to 18446744073709551616 [-Wimplicit-const-int-float-conversion]
  const double kUInt64ToProbabilityDivisor = std::numeric_limits<uint64_t>::max();
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/native/fuzzed_data_provider.cc:370:21: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
    for (int i = 0; i < count; ++i) {
                    ~ ^ ~~~~~
  4 warnings generated.
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/util.cc -o build/temp.linux-armv7l-3.9/src/native/util.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-z,relro -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.9/src/native/atheris.o build/temp.linux-armv7l-3.9/src/native/fuzzed_data_provider.o build/temp.linux-armv7l-3.9/src/native/util.o -o build/lib.linux-armv7l-3.9/atheris/native.cpython-39-arm-linux-gnueabihf.so
  building 'atheris.core_with_libfuzzer' extension
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/core.cc -o build/temp.linux-armv7l-3.9/src/native/core.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/counters.cc -o build/temp.linux-armv7l-3.9/src/native/counters.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-rsosxi4i/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/timeout.cc -o build/temp.linux-armv7l-3.9/src/native/timeout.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  src/native/timeout.cc:122:6: error: non-constant-expression cannot be narrowed from type 'long long' to '__time_t' (aka 'long') in initializer list [-Wc++11-narrowing]
      {timeout_secs / 2 + 1, 0}, { timeout_secs / 2 + 1, 0 }
       ^~~~~~~~~~~~~~~~~~~~
  src/native/timeout.cc:122:6: note: insert an explicit cast to silence this issue
      {timeout_secs / 2 + 1, 0}, { timeout_secs / 2 + 1, 0 }
       ^~~~~~~~~~~~~~~~~~~~
       static_cast<__time_t>( )
  src/native/timeout.cc:122:34: error: non-constant-expression cannot be narrowed from type 'long long' to '__time_t' (aka 'long') in initializer list [-Wc++11-narrowing]
      {timeout_secs / 2 + 1, 0}, { timeout_secs / 2 + 1, 0 }
                                   ^~~~~~~~~~~~~~~~~~~~
  src/native/timeout.cc:122:34: note: insert an explicit cast to silence this issue
      {timeout_secs / 2 + 1, 0}, { timeout_secs / 2 + 1, 0 }
                                   ^~~~~~~~~~~~~~~~~~~~
                                   static_cast<__time_t>( )
  2 errors generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for atheris
  Running setup.py clean for atheris

By changing:

-    {timeout_secs / 2 + 1, 0}, { timeout_secs / 2 + 1, 0 }
+    {static_cast<__time_t>(timeout_secs / 2 + 1), 0}, { static_cast<__time_t>(timeout_secs / 2 + 1), 0 }

The error seems to go away, but I get a linker err:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/Repositories/atheris
Building wheels for collected packages: atheris
  Building wheel for atheris (setup.py) ... /
/
error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-d8qayg97/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-d8qayg97/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-k04l70wj
       cwd: /tmp/pip-req-build-d8qayg97/
  Complete output (87 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-armv7l-3.9
  copying atheris_no_libfuzzer.py -> build/lib.linux-armv7l-3.9
  creating build/lib.linux-armv7l-3.9/atheris
  copying src/coverage_test_helper.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/version_dependent.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/regex_match_generation_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/coverage_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzzed_data_provider_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_mutator_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzz_test_lib.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/hook-atheris.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/import_hook.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/function_hooks.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/__init__.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/utils.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/pyinstaller_coverage_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_mutator_and_crossover_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/instrument_bytecode.py -> build/lib.linux-armv7l-3.9/atheris
  copying src/custom_crossover_fuzz_test.py -> build/lib.linux-armv7l-3.9/atheris
  running build_ext
  Your libFuzzer version is too old, but it's possible to attempt an in-place upgrade. Trying that now.
  Your libFuzzer is up-to-date.
  creating tmp
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.9 -c /tmp/tmpk1dbls7t.cpp -o tmp/tmpk1dbls7t.o -std=c++14
  building 'atheris.native' extension
  creating build/temp.linux-armv7l-3.9
  creating build/temp.linux-armv7l-3.9/src
  creating build/temp.linux-armv7l-3.9/src/native
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/atheris.cc -o build/temp.linux-armv7l-3.9/src/native/atheris.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/fuzzed_data_provider.cc -o build/temp.linux-armv7l-3.9/src/native/fuzzed_data_provider.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  src/native/fuzzed_data_provider.cc:161:23: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
      for (int i = 0; i < bytes; ++i) {
                      ~ ^ ~~~~~
  src/native/fuzzed_data_provider.cc:212:23: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
      for (int i = 0; i < bytes; ++i) {
                      ~ ^ ~~~~~
  src/native/fuzzed_data_provider.cc:284:44: warning: implicit conversion from 'unsigned long long' to 'const double' changes value from 18446744073709551615 to 18446744073709551616 [-Wimplicit-const-int-float-conversion]
  const double kUInt64ToProbabilityDivisor = std::numeric_limits<uint64_t>::max();
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  src/native/fuzzed_data_provider.cc:370:21: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
    for (int i = 0; i < count; ++i) {
                    ~ ^ ~~~~~
  4 warnings generated.
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=native -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/util.cc -o build/temp.linux-armv7l-3.9/src/native/util.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-z,relro -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.9/src/native/atheris.o build/temp.linux-armv7l-3.9/src/native/fuzzed_data_provider.o build/temp.linux-armv7l-3.9/src/native/util.o -o build/lib.linux-armv7l-3.9/atheris/native.cpython-39-arm-linux-gnueabihf.so
  building 'atheris.core_with_libfuzzer' extension
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/core.cc -o build/temp.linux-armv7l-3.9/src/native/core.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/counters.cc -o build/temp.linux-armv7l-3.9/src/native/counters.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/timeout.cc -o build/temp.linux-armv7l-3.9/src/native/timeout.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/tracer.cc -o build/temp.linux-armv7l-3.9/src/native/tracer.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  src/native/tracer.cc:65:21: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned int') [-Wsign-compare]
    for (int i = 0; i < n; ++i) {
                    ~ ^ ~
  1 warning generated.
  /usr/bin/clang -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DVERSION_INFO='2.0.12' -DATHERIS_MODULE_NAME=core_with_libfuzzer -I/tmp/pip-req-build-d8qayg97/.eggs/pybind11-2.10.0-py3.9.egg/pybind11/include -I/usr/include/python3.9 -c src/native/util.cc -o build/temp.linux-armv7l-3.9/src/native/util.o -Wno-deprecated-declarations -Wno-attributes -std=c++14
  /usr/bin/clang++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-z,relro -g -fwrapv -O2 -g -ffile-prefix-map=/python3.9-3.9.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-armv7l-3.9/src/native/core.o build/temp.linux-armv7l-3.9/src/native/counters.o build/temp.linux-armv7l-3.9/src/native/timeout.o build/temp.linux-armv7l-3.9/src/native/tracer.o build/temp.linux-armv7l-3.9/src/native/util.o -o build/lib.linux-armv7l-3.9/atheris/core_with_libfuzzer.cpython-39-arm-linux-gnueabihf.so /tmp/tmp.wm7rnraAGA.a
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o)(.text+0x18fc): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o): in function `fuzzer::TracePC::RecordInitialStack()':
  (.text+0x18fc): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o)(.text+0x1920): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o): in function `fuzzer::TracePC::GetMaxStackOffset() const':
  (.text+0x1920): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o)(.text._ZTW21__sancov_lowest_stack[_ZTW21__sancov_lowest_stack]+0x14): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerTracePC.o): in function `TLS wrapper function for __sancov_lowest_stack':
  (.text._ZTW21__sancov_lowest_stack[_ZTW21__sancov_lowest_stack]+0x14): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o)(.text+0x7e0): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o): in function `fuzzer::Fuzzer::Fuzzer(int (*)(unsigned char const*, unsigned int), fuzzer::InputCorpus&, fuzzer::MutationDispatcher&, fuzzer::FuzzingOptions)':
  (.text+0x7e0): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o)(.text+0xd54): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o): in function `fuzzer::Fuzzer::AlarmCallback()':
  (.text+0xd54): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o)(.text+0x2294): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o): in function `fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned int)':
  (.text+0x2294): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o)(.text+0x2518): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o): in function `fuzzer::Fuzzer::GetCurrentUnitInFuzzingThead(unsigned char const**) const':
  (.text+0x2518): dangerous relocation: unsupported relocation
  /usr/bin/ld: /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o)(.text._ZTWN6fuzzer6Fuzzer10IsMyThreadE[_ZTWN6fuzzer6Fuzzer10IsMyThreadE]+0x14): R_ARM_TLS_LE32 relocation not permitted in shared object
  /tmp/tmp.wm7rnraAGA.a(FuzzerLoop.o): in function `TLS wrapper function for fuzzer::Fuzzer::IsMyThread':
  (.text._ZTWN6fuzzer6Fuzzer10IsMyThreadE[_ZTWN6fuzzer6Fuzzer10IsMyThreadE]+0x14): dangerous relocation: unsupported relocation
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command '/usr/bin/clang++' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for atheris
  Running setup.py clean for atheris
Failed to build atheris

Any suggestions would be most appreciated!

TheShiftedBit commented 2 years ago

It looks like those errors are being caused by libFuzzer being linked as a shared library in Atheris, as opposed to being statically linked into the executable.

I'm not very familiar with ARM, so I don't know what would be needed to allow libFuzzer to be linked dynamically.

However, it's possible you can avoid this problem by linking libFuzzer directly into Python. Follow these instructions: https://github.com/google/atheris/blob/master/native_extension_fuzzing.md#option-2-linking-libfuzzer-into-python Then, modify Atheris' setup.py to remove the atheris.core_with_libfuzzer extension (you want the atheris.core_without_libfuzzer one) and remove everything in build_extensions() after # Deploy versions of ASan and UBSan that have been merged with libFuzzer.

chrisruk commented 1 year ago

Thanks for your help I read through the native extension doc and re-read the readme, and tried compiling llvm to see if that fixed my issue.

In case anyone else wants to install atheris on ARM/Raspberry Pi, I did -

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS='clang;compiler-rt' -G "Unix Makefiles" ../llvm
make -j 4 # This step is very slow
sudo make install
git clone https://github.com/google/atheris.git
cd atheris
CLANG_BIN="/usr/local/bin/clang" LIBFUZZER_LIB="/usr/local/lib/clang/16.0.0/lib/linux/libclang_rt.fuzzer_no_main-armhf.a" pip3 install .

When doing a simple test, I got -

ImportError: /home/pi/.local/lib/python3.9/site-packages/atheris/core_with_libfuzzer.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_load_8

This could be fixed by doing -

LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1 python3.9 ex.py

Look forward to playing with atheris now!

stulle123 commented 1 year ago

Hey @chrisruk after following your instructions I'm running into an Illegal Instruction error:

$ LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1 python3.9 fuzz.py
INFO: Using built-in libfuzzer
Illegal instruction

Any idea on what is going wrong?

I'm using a Pi Zero armv6 with clang 13.0.1.