trailofbits / ruzzy

A coverage-guided fuzzer for pure Ruby code and Ruby C extensions
GNU Affero General Public License v3.0
78 stars 5 forks source link

Fix #7, add coverage support for Ruby code #8

Closed mschwager closed 7 months ago

mschwager commented 7 months ago

Getting this working took quite a bit of hackery, but it's working 🎉. All the links I posted in #7 should help with some context. This also took a significant amount of reading and understanding the Ruby source code.

Fortunately, Ruby has a builtin mechanism for tracking coverage information. The problem is that this functionality isn't available as part of the public C API. RUBY_EVENT_COVERAGE_BRANCH exists, but it's internal-only. Again, fortunately we can specify it and the Ruby internals will still respect its event hooking. Perhaps we can file an issue with Ruby asking them to make this functionality part of their public API. Another problem is the coverage tracking cannot be turned on via the public C API, at least that I could find. So we have to manually call its start functionality. Another oddity is that it needs to require a separate script. I never did figure out why this is the case, but I replicated it to make it work.

Here's an example of Ruby coverage support:

test_trace.rb:

# frozen_string_literal: true

require 'ruzzy'

Ruzzy.c_trace_branch

require_relative 'test_ruby.rb'

test_ruby.rb:

# frozen_string_literal: true

require 'ruzzy'

# Exercises c_trace_branch
test_one_input_1 = lambda do |data|
  if data.length == 4
    if data[0] == 'F'
      if data[1] == 'U'
        if data[2] == 'Z'
          if data[3] == 'Z'
            raise
          end
        end
      end
    end
  end
  return 0
end

# Exercises c_trace_cmp8
test_one_input_2 = lambda do |data|
  if data.unpack('H*').first.to_i(16) === "FUZZ".unpack('H*').first.to_i(16)
    raise
  end
  return 0
end

# Exercises c_trace_div8
test_one_input_3 = lambda do |data|
  100 / data.unpack('H*').first.to_i(16)
  return 0
end

Ruzzy.fuzz(test_one_input_1)
$ LD_PRELOAD=$(ruby -e 'require "ruzzy"; print Ruzzy::ASAN_PATH') ruby -Ilib test_trace.rb
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 2220906844
INFO: Loaded 1 modules   (256 inline 8-bit counters): 256 [0xffffb4371218, 0xffffb4371318), 
INFO: Loaded 1 PC tables (256 PCs): 256 [0xffffb4370218,0xffffb4371218), 
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
...
SUMMARY: libFuzzer: fuzz target exited
MS: 1 CopyPart-; base unit: 9971ed5bd9798b4f7ba42b4e2f51cd073fcad39c
0x46,0x55,0x5a,0x5a,
FUZZ
artifact_prefix='./'; Test unit written to ./crash-aea2e3923af219a8956f626558ef32f30a914ebc
Base64: RlVaWg==