RSpec::Core::Runner.run duplicates specs when run twice from within ruby

[DataWraith]

I am invoking RSpec from within ruby (as seen in #359). After it finishes, the specs and results are not discarded. Invoking RSpec::Core::Runner.run repeatedly yields output with the specs doubled/tripled/etc. Is there a way to make RSpec discard the results of a run in order to start fresh at the next invocation?

EDIT: It's not only the specs that are doubled, the results are as well. I think this might be because the test/implementation files have already been required and are thus not reloaded by the second invocation.

My use case is that I want to notify the user that something went wrong via growl/libnotify when s/he is doing BDD. To do that, I need the last Exception RSpec raised (if any), and I found wrapping RSpec::Core::Runner.run into begin, rescue, end to be the simplest way to achieve this.

Demonstration:

$ rspec --version
2.5.2
$ irb
>> require 'rspec'
=> true
>> RSpec::Core::Runner.run(['spec'], STDERR, STDOUT)

Ascii85
  #encode
    should [...]
  #decode
    should [...]

Finished in 0.07314 seconds
14 examples, 0 failures
=> true
>> RSpec::Core::Runner.run(['spec'], STDERR, STDOUT)
/home/datawraith/projects/Ascii85/spec/lib/ascii85_spec.rb:7: warning: already initialized constant TEST_CASES

Ascii85
  #encode
    should [...]
  #decode
    should [...]

Ascii85
  #encode
    should [...]
  #decode
    should [...]

Finished in 0.09018 seconds
42 examples, 0 failures
=> true

[dchelimsky]

As a temporary workaround (while I figure out a real solution), do this before calling RSpec::Core::Runner.run:

RSpec::Core::Runner.disable_autorun!

Then, after each call to RSpec::Core::Runner.run:

RSpec.instance_variable_set("@world",nil)
RSpec.instance_variable_set("@configuration",nil)

[DataWraith]

Wow, that was quick! Thanks for investigating this!

In case it helps you zero in on the real solution: The workaround works partially for me. There is no doubling anymore, but the results stay the same, even if I introduce a syntax error into the tested code, which should raise an Exception. Probably for the same reason, spec_helper.rb and the included RSpec.configure { } block don't get executed after the configuration is nuked.

Let me know if there is anything I can do to help.

[dchelimsky]

Working in an irb session I can see changes to spec or implementation files have an impact on the result, so I'm not sure why you're not seeing that. RSpec does use load to load the spec files, so it should reload them the 2nd time around.

As for spec_helper, it's probably being required from your specs (since that is the convention), but for this to work you'll have to use load instead of require:

load "spec_helper.rb"

[DataWraith]

Oh, yeah. That fixes it completely!

I'm not 100% sure what causes the results to 'stick', but it doesn't occur when loading the spec_helper instead of requiring it. When I require it, the second invocation of rspec runs all specs, not only the ones for the current ruby version (tag :ruby => 1.{8,9}) because the configuration block is not executed. After that the results seem to get stuck somehow. Anyway, it's not a problem with load.

Thanks again for your help!

[dchelimsky]

bfd07650f54c1b432d03cffaadcf9d3cab2bf273