search

rheostat2718 / unladen-swallow

Automatically exported from code.google.com/p/unladen-swallow
Other
0 stars 0 forks source link

Add fuzz-based testing #15

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
As we do deeper and deeper surgery on the compiler, I'd like to have a
fuzzer or some other kind of randomized testing to look for corner cases in
our implementation.

Before writing our own, we should try to reuse Fusil
(http://fusil.hachoir.org/svn/trunk/), which has been shown to find bugs in
CPython already. Other Python implementations may already have something
like this; if so, we should reuse that.

Eventually, this would become part of the continuous build, forever
searching the space of random Python programs for crashes.

Original issue reported on code.google.com by collinw on 14 Apr 2009 at 11:46

GoogleCodeExporter commented 9 years ago 
Additional ideas: http://www.squarefree.com/2008/12/23/fuzzing-tracemonkey/,
including using fuzzers to identify performance corner cases.

Original comment by collinw on 15 Apr 2009 at 12:41

GoogleCodeExporter commented 9 years ago

Original comment by collinw on 27 May 2009 at 11:07

GoogleCodeExporter commented 9 years ago

Original comment by collinw on 29 May 2009 at 12:09

GoogleCodeExporter commented 9 years ago

Original comment by collinw on 29 May 2009 at 4:13

GoogleCodeExporter commented 9 years ago 
Fusil is primarily for testing extension modules; pyfuzz 
(http://bytebucket.org/ebo/pyfuzz/) is probably a better tool for fuzz testing 
the 
changes we're making to the compiler.

Original comment by collinw on 15 Dec 2009 at 6:46

GoogleCodeExporter commented 9 years ago 
At PyCon 2010, Guido requested that we develop and deploy a fuzzer that targets 
the bytecode->LLVM IR 
compiler to look for possible security holes/crashers. This will block final 
merger with py3k.

Original comment by collinw on 22 Mar 2010 at 6:14

GoogleCodeExporter commented 9 years ago

Original comment by collinw on 19 May 2010 at 7:02

GoogleCodeExporter commented 9 years ago 
r1159 added a JIT fuzzing tool and fixed numerous segfaults/aborts/asserts 
uncovered by said tool. I'm lowering the priority accordingly, since this 
should satisfy Guido's request.

I was only able to uncover problems in the JIT compiler itself, nothing in 
LLVM. The restrictions on the incoming bytecode are sufficiently tight that I 
wasn't able to generate code that would snarl LLVM (make it slow perhaps, but 
no crashes).

Original comment by collinw on 15 Jun 2010 at 8:33