search

jerryscript-project / jerryscript

Ultra-lightweight JavaScript engine for the Internet of Things.
https://jerryscript.net
Apache License 2.0
6.95k stars 673 forks source link

Stress testing of the engine #333

Closed egavrin closed 8 years ago

egavrin commented 9 years ago

Do you know any fuzzers or stress testing suites that can useful for us?

akosthekiss commented 9 years ago

The University of Szeged has a proprietary fuzz testing framework with no imminent plans for open sourcing but is happy to share any issues it discovers in Jerry. (Actually, several previously reported issues were discovered with its help.)

renatahodovan commented 9 years ago

The "problem" with jsfunfuzz is that it generates the test cases on the fly which means that the whole generator script needs to be loaded by the engine every time. Hence jerry would need to load ~5500 LOC to generate a single test case but unfortunately it runs of the registers quickly.

I've also tried the recently popular afl fuzzer against jerry however it constantly crashes on the longjmp bug (#341) and it's unusable this way.

ruben-ayrapetyan commented 9 years ago

@renatahodovan, maybe we can generate tests by running jsfunfuzz separately, and then use the generated tests for checking.


I've also tried the recently popular afl fuzzer against jerry however it constantly crashes on the longjmp bug (#341) and it's unusable this way.

Could you, please, check #390?

ruben-ayrapetyan commented 9 years ago

341 is fixed.

egavrin commented 9 years ago

@renatahodovan could you please share how to run afl-fuzz for javascript?

renatahodovan commented 9 years ago

@egavrin sure, however I'm also still working on how to exploit all of its features.

A customized version of afl-fuzz is available from here: https://github.com/tunz/afl-fuzz-js You can run it two different ways: in the first case you can use your own jerry build, choose a set of initial test cases that will be mutated and a predefined dictionary to guide the mutation and go ahead. From the manual, starting a fuzz session looks something like this:

/path/to/afl-fuzz -i [input directory] -o [output directory] -x [dictionary directory] -m 8G ./jerry @@

I've ran this for days but it hasn't found much (except the longjump crashes).

However, there is second coverage driven option (what is said much more efficient). The problem is that to use this option the target must be instrumented with the afl-fuzzer's own wrapped gcc/g++. Unfortunately, replacing jerry's default compiler is not seems too easy. I've achieved to build everything with afl-g++, but it still fails on linking. This is what I'm experimenting with right now :)

akosthekiss commented 9 years ago

There is an experimental branch in my repo for overriding the default compiler (https://github.com/akiss77/jerryscript/commits/compiler-override) but it's not ready for PR yet. Feel free to (ab)use it directly from there though. (However, even that experiences the linker issue.)