Is there any interest in prebuilt distributions?

[pschanely]

Building LLVM is painful! I could experiment with "Github actions" to try and get wheels building for the popular platforms if you'd like.

@Zac-HD FYI - you might be interested in this too?

[TheShiftedBit]

This would actually be fairly easy. The main reason we didn't is that when fuzzing native code, the version of libFuzzer needs to roughly correspond to the version of Clang used to build the extensions.

However, that doesn't matter for people only fuzzing Python. @inferno-chromium Do you have an opinion?

[inferno-chromium]

This would actually be fairly easy. The main reason we didn't is that when fuzzing native code, the version of libFuzzer needs to roughly correspond to the version of Clang used to build the extensions.

However, that doesn't matter for people only fuzzing Python. @inferno-chromium Do you have an opinion?

Yes right, prebuilts does not help to match clang version with one used to build extension. And you need to build extension anyway locally with some clang, so can't escape from that. Do you really need to build llvm from source, why not use prebuilt clang packages ?

[TheShiftedBit]

Atheris will already attempt to use the installed Clang. That doesn't work on the Clang that comes with XCode (it doesn't include libFuzzer), but typically works on Linux.

On Tue, Feb 16, 2021, 2:49 PM Abhishek Arya notifications@github.com wrote:

This would actually be fairly easy. The main reason we didn't is that when fuzzing native code, the version of libFuzzer needs to roughly correspond to the version of Clang used to build the extensions.

However, that doesn't matter for people only fuzzing Python. @inferno-chromium https://github.com/inferno-chromium Do you have an opinion?

Yes right, prebuilts does not help to match clang version with one used to build extension. And you need to build extension anyway locally with some clang, so can't escape from that. Do you really need to build llvm from source, why not use prebuilt clang packages ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/atheris/issues/14#issuecomment-780108996, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWZFLUB7NQ4LB5Q4CMMCTS7LK7FANCNFSM4XWX5JTQ .

[pschanely]

FWIW, my use case is for pure python. Namely, I'm trying to make CrossHair be able to work alongside a fuzzer and share corpus examples in both directions. Making it trivial for people to install the fuzzer alongside CrossHair is really important, however.

I guess if you'd prefer to encourage direct atheris users to build locally, I could also just build a fork and publish it separately on PyPI?

[pschanely]

Friendly ping on this. 😄 Do you have concerns or recommendations about a separate prebuilt distribution on PyPI, if it was named appropriately? ("atheris-pure-python"?)

[TheShiftedBit]

I'll make some prebuilt distributions available, either in the same package or a separate one. Expect this done within a few days.

[TheShiftedBit]

Hi,

I've published prebuilt binary packages to https://test.pypi.org/project/atheris/1.0.13/ for Linux 64 bit, Python 3.5-3.9. (That's test PyPI). Could you test them out and see if they work reasonably? Deployment code is also pushed to the prebuilt branch.

[pschanely]

Awesome! I've confirmed pip installation locally (via docker, I'm on mac):

>>> import atheris
>>> atheris.__file__
'/usr/local/lib/python3.8/site-packages/atheris.cpython-38-x86_64-linux-gnu.so'

WDYT about mac/windows? I've seen other folks have success with cibuildwheel in github actions. Would be happy to experiment myself a little if that would help!

[TheShiftedBit]

Indeed! I plan to offer Mac builds as well. That's actually where they would do the most good, since you have to install Clang from source on Macs (but typically not on Linux).

I was planning to build the Mac release separately and copy it over, but if you want to take a stab at Github actions, go for it! I won't be able to do Mac builds until next week.

On Wed, Apr 28, 2021, 8:24 AM Phillip Schanely @.***> wrote:

Awesome! I've confirmed pip installation locally (via docker, I'm on mac):

import atheris atheris.file '/usr/local/lib/python3.8/site-packages/atheris.cpython-38-x86_64-linux-gnu.so'

WDYT about mac/windows? I've seen other folks have success with cibuildwheel in github actions https://github.com/joerick/cibuildwheel#example-setup. Would be happy to experiment myself a little if that would help!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/google/atheris/issues/14#issuecomment-828452226, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKWZFMC7R363LXLS4PIJY3TLAEAFANCNFSM4XWX5JTQ .

[pschanely]

So, I didn't quite get cibuildwheel to work, but a simple script in this GitHub action builds for me! Feel free to pull it in if you like (it would be manually triggered under the Actions tab, and you'd then manually upload the wheel to PyPI, though either of those could be automated too)

I think I hadn't noticed originally that Windows isn't supported. Apparently, libfuzzer does work on Windows: the main blockers would be the various shell scripts and such that we use during setup? Or is there a more fundamental problem that would prevent this from working?

[TheShiftedBit]

Thanks for the GitHub action!

Prebuilt releases are now on PyPI for both Linux and Mac. I didn't use the GitHub action yet because I don't want to build Linux with a script and Mac with a Github action; I'd rather they be consistent. However, I'll probably need to switch to entirely Github actions eventually, to best support M1 Macs other Mac versions.

[TheShiftedBit]

Oh, and as for Windows: nope, nothing stopping us from supporting Windows. I didn't put any effort into it because Google rarely uses Windows and Python is rarely used on Windows, but it's possible. PRs accepted. :)

I'll probably switch entirely to GitHub actions sometime soon.