BitFunnel / NativeJIT

A C++ expression -> x64 JIT
http://bitfunnel.org/
MIT License
1.14k stars 85 forks source link

Non-deterministic behavior from Parser #68

Open danluu opened 8 years ago

danluu commented 8 years ago
                       american fuzzy lop 2.33b (Parser)

┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐
│        run time : 0 days, 0 hrs, 13 min, 10 sec      │  cycles done : 2      │
│   last new path : 0 days, 0 hrs, 1 min, 7 sec        │  total paths : 429    │
│ last uniq crash : 0 days, 0 hrs, 1 min, 43 sec       │ uniq crashes : 9      │
│  last uniq hang : none seen yet                      │   uniq hangs : 0      │
├─ cycle progress ────────────────────┬─ map coverage ─┴───────────────────────┤
│  now processing : 237* (55.24%)     │    map density : 0.54% / 3.18%         │
│ paths timed out : 0 (0.00%)         │ count coverage : 5.08 bits/tuple       │
├─ stage progress ────────────────────┼─ findings in depth ────────────────────┤
│  now trying : havoc                 │ favored paths : 43 (10.02%)            │
│ stage execs : 112/512 (21.88%)      │  new edges on : 66 (15.38%)            │
│ total execs : 1.08M                 │ total crashes : 15 (9 unique)          │
│  exec speed : 1434/sec              │   total hangs : 0 (0 unique)           │
├─ fuzzing strategy yields ───────────┴───────────────┬─ path geometry ────────┤
│   bit flips : 43/26.2k, 12/26.1k, 4/26.0k           │    levels : 7          │
│  byte flips : 0/3278, 2/3044, 3/2903                │   pending : 345        │
│ arithmetics : 15/173k, 0/16.2k, 0/2032              │  pend fav : 0          │
│  known ints : 2/18.0k, 0/82.7k, 1/127k              │ own finds : 426        │
│  dictionary : 0/0, 0/0, 2/2810                      │  imported : n/a        │
│       havoc : 351/563k, 0/0                         │ stability : 99.86%     │
│        trim : 7.76%/1098, 4.50%                     ├────────────────────────┘
└─────────────────────────────────────────────────────┘             [cpu: 23%]

Note that stabillity is < 100%, which means that the inputs aren't producing deterministic outputs. I don't think we have anything that's purposely non-deterministic and we're not multi-threaded, so I wonder if we're somehow using uninitialized memory?

danluu commented 8 years ago

This goes away if I limit the expression length so that the arena allocator doesn't run out of memory. This seems to indicate that we have non-deterministic behavior when we run out of memory. That shouldn't happen, but since we haven't really tested error cases it wouldn't be too surprising if we do something bad in the error case.

danluu commented 8 years ago

I checked this with valgrind because it's pretty good about finding uses of uninitialized memory and it didn't find anything even in the case where we blow up the allocator. Considering how minor this is, I don't think is worth pursuing further right now, but I am pretty curious about what's going on here.