Potential file conflict with other Python packages

[grawlinson]

When packaging this for Arch Linux, I noticed that there is the potential for file conflicts with other Python packages.

Here's the contents of the 3.10 wheel from PyPi (which corresponds with a distribution-level Python package):

atheris-2.0.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

```sh atheris_no_libfuzzer.py ubsan_with_fuzzer.so asan_with_fuzzer.so ubsan_cxx_with_fuzzer.so libclang_rt.fuzzer_no_main-x86_64.a atheris-2.0.12.dist-info/WHEEL atheris-2.0.12.dist-info/RECORD atheris-2.0.12.dist-info/top_level.txt atheris-2.0.12.dist-info/LICENSE atheris-2.0.12.dist-info/entry_points.txt atheris-2.0.12.dist-info/METADATA atheris/instrument_bytecode.py atheris/native.cpython-310-x86_64-linux-gnu.so atheris/custom_mutator_and_crossover_fuzz_test.py atheris/pyinstaller_coverage_test.py atheris/import_hook.py atheris/version_dependent.py atheris/fuzzed_data_provider_test.py atheris/coverage_test.py atheris/utils.py atheris/fuzz_test_lib.py atheris/custom_mutator_fuzz_test.py atheris/custom_mutator.cpython-310-x86_64-linux-gnu.so atheris/custom_crossover.cpython-310-x86_64-linux-gnu.so atheris/regex_match_generation_test.py atheris/function_hooks.py atheris/fuzz_test.py atheris/__init__.py atheris/core_without_libfuzzer.cpython-310-x86_64-linux-gnu.so atheris/hook-atheris.py atheris/custom_crossover_fuzz_test.py atheris/coverage_test_helper.py atheris/core_with_libfuzzer.cpython-310-x86_64-linux-gnu.so ```

Of interest are these particular files at the package root:

• ubsan_with_fuzzer.so
• asan_with_fuzzer.so
• ubsan_cxx_with_fuzzer.so
• libclang_rt.fuzzer_no_main-x86_64.a

I'm wondering if these are leftover build artifacts, are you able to confirm if these are supposed to be present?

[TheShiftedBit]

The three .so files are intentionally generated and should be present. They are useful to users when running sanitizers on native code. Without them, there is a conflict between some of the symbols in libFuzzer and the sanitizers unless they are linked together into the same object.

The .a file is indeed a leftover from the build process. I'll see about removing it.

On Mon, Oct 10, 2022, 5:25 PM George Rawlinson @.***> wrote:

When packaging this for Arch Linux, I noticed that there is the potential for file conflicts with other Python packages.

Here's the contents of the 3.10 wheel from PyPi (which corresponds with a distribution-level Python package): atheris-2.0.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

atheris_no_libfuzzer.py ubsan_with_fuzzer.so asan_with_fuzzer.so ubsan_cxx_with_fuzzer.so libclang_rt.fuzzer_no_main-x86_64.a atheris-2.0.12.dist-info/WHEEL atheris-2.0.12.dist-info/RECORD atheris-2.0.12.dist-info/top_level.txt atheris-2.0.12.dist-info/LICENSE atheris-2.0.12.dist-info/entry_points.txt atheris-2.0.12.dist-info/METADATA atheris/instrument_bytecode.py atheris/native.cpython-310-x86_64-linux-gnu.so atheris/custom_mutator_and_crossover_fuzz_test.py atheris/pyinstaller_coverage_test.py atheris/import_hook.py atheris/version_dependent.py atheris/fuzzed_data_provider_test.py atheris/coverage_test.py atheris/utils.py atheris/fuzz_test_lib.py atheris/custom_mutator_fuzz_test.py atheris/custom_mutator.cpython-310-x86_64-linux-gnu.so atheris/custom_crossover.cpython-310-x86_64-linux-gnu.so atheris/regex_match_generation_test.py atheris/function_hooks.py atheris/fuzz_test.py atheris/init.py atheris/core_without_libfuzzer.cpython-310-x86_64-linux-gnu.so atheris/hook-atheris.py atheris/custom_crossover_fuzz_test.py atheris/coverage_test_helper.py atheris/core_with_libfuzzer.cpython-310-x86_64-linux-gnu.so

Of interest are these particular files at the package root:

• ubsan_with_fuzzer.so
• asan_with_fuzzer.so
• ubsan_cxx_with_fuzzer.so
• libclang_rt.fuzzer_no_main-x86_64.a

I'm wondering if these are leftover build artifacts, are you able to confirm if these are supposed to be present?

— Reply to this email directly, view it on GitHub https://github.com/google/atheris/issues/46, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWZFLEDZXA7XGMKOST3WDWCSCVXANCNFSM6AAAAAARBWC5GE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[TheShiftedBit]

Actually, it turns out the libclang_rt.fuzzer_no_main-x86_64.a is also intentionally present. It's the version of libfuzzer that was used to build Atheris, and is included to allow building CPython with libFuzzer integrated into it. libFuzzer versions more than a couple years old aren't compatible with Atheris without modifications, so we can't rely on the system-installed libFuzzer version.

As for why these are at the package root: historical reasons, unfortunately. An earlier version of Atheris was implemented entirely in native code, and so the package was nothing but a .so file; there was no atheris subdirectory to put the fuzzer .so files in. Now we have a directory, but if we moved the files, it would probably break a bunch of users' fuzzing infrastructure.

Do you believe these files are likely to cause conflicts with other packages? At the moment, the compatibility balance is probably toward not moving them, but if there are other packages with files by the same name, I could see that changing.

[grawlinson]

To be honest the odds of a file conflict are quite low. Feel free to close this issue though. :)