Question: Corpus test?

[mbj]

Hello regexp_parser team!

While improving mutants regexp support I'm often challenged with "Did I cover all nodes?" type of questions, and for that reason I try to source as many test cases as possible from my dependencies. I did so in unparser re-using the parser test suite for ruby edge cases.

And I'd love if I had the ability to do this for the regexp parser dependency also. Is there a good way to 'mass source' expressions from your libraries tests?

Cheers,

Markus

[jaynetics]

hi!

i think this is a good idea. the regexp_parser specs include quite a few edge cases that are neither in the Onigmo specs nor in ruby-spec.

my first thought was that you could parse the spec files and extract all regexps from the ASTs, but this won't work as many are only represented as strings - else they would emit warnings or cause SyntaxErrors on older Ruby versions.

maybe you could do something like this:

$regexps = []

Regexp::Scanner.prepend(Module.new do
  def scan(input, options:, &block)
    begin
      $regexps << (input.is_a?(Regexp) ? input : Regexp.new(input, options))
    rescue SyntaxError, RegexpError
      # invalid regexp, purposefully tested or due to Ruby version
    end

    super
  end
end)

# run specs

this attaches to the lowest level of the library, so it will catch all Regexps processed in the specs.

it will emit some warnings and the size of the resulting Array will differ depending on which Ruby version it is run on.

[mbj]

I wounder if you would support a refactoring moving the test cases to data and running through that data via rspec? That data would than indicate support levels etc. parser does the same but on minitest and the custom runnier I cited above is able to extract the data. This is a bit more hairy with rspec as hooking it is not 'that' easy.

[jaynetics]

i'd support this if readability of the specs is maintained.

it might be a major undertaking though, because the scope of this gem goes a bit beyond mere AST generation, and the specs are not super uniform.

an intermediate approach could perhaps look at the shared examples 'scan', 'lex' and 'parse', either by hooking into them (in a hairy fashion) or parsing their include_examples callsites. they all take a Regexp or regexp-like String as first argument.

[mbj]

I cannot tell you if/when I get to this. Lets close this for now as my question was answered.