search

rui314 / 8cc

A Small C Compiler
MIT License
6.13k stars 740 forks source link

72 crashing files #49

Open jodiecunningham opened 9 years ago

jodiecunningham commented 9 years ago

Hi rui314,

I have been fuzzing your compiler and discovered a few crashers to submit.

Reproducer cmdline: 8cc -c $reproducer

In the tarball I have provided a gdblog for each reproducer. There are a total of 72 crashers. I'm not sure that there are not duplicates in this set, but I got this distilled from 1000

The version I am using is from commit 11dd5a567b63995bda235d46e5aea5a3e615b845 System: AMD64 Ubuntu 14.04

Found using the fuzzer American Fuzzy Lop ( http://lcamtuf.coredump.cx/afl/ )

Regards, -Jodie

jodiecunningham commented 9 years ago

Tarball here: https://www.dropbox.com/s/s7u539kvdflpwde/8cc_crashers_20150304.tgz?dl=0

andrewchambers commented 9 years ago

Thanks for your efforts.

In the future its probably easiest to fuzz using https://embed.cs.utah.edu/csmith/ which is designed specifically for fuzzing compilers by generating random valid programs. The C test case reducer http://embed.cs.utah.edu/creduce/ can then turn the failing tests into a few lines automatically.

There is no shortage of crashes, Its possible to find many simply by running the gcc test suite. I think the limiting factor is programmer time (this is a hobby project), not bugs at the moment.

jodiecunningham commented 9 years ago

@andrewchambers I don't mean to discount the usefulness of csmith here, as I will certainly look into using it in parallel to afl-fuzz, but one of the benefits of afl-fuzz is that it will find bugs by generating crashers that are not otherwise valid programs. :)

andrewchambers commented 9 years ago

Of course, but I don't really see why anyone cares about crashes in programs that a person would not manually write by hand.

  1. The compiler is unprivileged.
  2. You should audit source code you download from the internet if it is not trusted.
  3. If you do not audit the source code, it would be easier to just put malware in the source code than to exploit the compiler itself. The next logical step after compiling a program is running it anyway.
  4. You could just as easily put an evil script in the makefile.
  5. If you are building code from an untrusted source, you should have a sandboxed environment.

Edit: In general, people using a compiler are cooperating with the compiler, not an adversary. If someone finds something which crashes a compiler while writing code, they generally refactor the code so it no longer crashes so they can continue working (while hopefully saving a test case reduced with creduce).

andrewchambers commented 9 years ago

One more note - miscompilation bugs are generally the most valuable for a compiler. If code is accepted by a compiler, but the code produces an incorrect result, this is a serious issue. Crashes can be detected by the end user far more easily.

Csmith actively finds miscompilation if you compare output to other compilers. Crash bugs are important, though less important.

Don't be discouraged though, keep fuzzing :), just keep these points in mind.

jodiecunningham commented 9 years ago

From my perspective I assume that it may be possible for an error in a user-generated file to generate one of these crashes, and for those people I'm sure they'd rather have a useful error message than a segfault. My focus for this fuzzing wasn't related to security. After all, it's a compiler.

I caught this on HN, and I sometimes when I catch these projects I try to do some fuzzing for them. Big projects get a lot of effort and manpower, but the small projects don't get a lot of feedback. So I try to pitch in. It's maybe 10 minutes work I've spent to generate these.

andrewchambers commented 9 years ago

That is true, I imagine many of these are caused by some underlying bug or code smell anyway. It may take a while before we can fix them all haha.