parse the last line of the block only on failure????

[dchelimsky]

So I have this kinda crazy idea for integrating wrong with rspec:

describe User do
  it "has the read-only role by default" do
    User.new.role == Role::READ_ONLY
  end
end

The idea is that, if configured to do so, rspec would forward the block passed to it to assert (or maybe wrong could expose a function to support similar w/o having to include anything). This way you don't even need to fuss over "assert" v "expect" - the block passed to it has to conform to the same rules as a block passed to wrong's assert.

As a preliminary experiment, I tried this:

require 'wrong/adapters/rspec'

describe "something" do
  it "does something" do
    assert do
      x = 5
      y = 3
      x == y
    end
  end

  it "does something else" do
    x = 5
    y = 3
    assert { x == y }
  end
end

When I ran this I got the following output:

FF

Failures:

  1) something does something
     Failure/Error: assert do
       Expected x = 5
       y = 3
       (x == y)
       , but
           (x == y) raises NameError: undefined local variable or method `x' for #<RSpec::Core::ExampleGroup::Nested_1:0x007fe41bdd35e0>
           x raises NameError: undefined local variable or method `x' for #<RSpec::Core::ExampleGroup::Nested_1:0x007fe41bdd35e0>
           y raises NoMethodError: undefined method `yaml' for nil:NilClass
     # ./example_spec.rb:5:in `block (2 levels) in <top (required)>'

  2) something does something else
     Failure/Error: assert { x == y }
       Expected (x == y), but
           x is 5
           y is 3
     # ./example_spec.rb:15:in `block (2 levels) in <top (required)>'

As you can see, wrong does a great job of providing feedback for the example with the single line block, but the feedback for the multi-line block is a bit useless. So the question is: can wrong be modified to only care about the last statement in the block? Of course, the other question is: do you think this idea is interesting enough to try to make it happen?

[justinko]

OMG this would be amazing.

[myronmarston]

This is a really interesting idea, but I do have a couple concerns...

• It's very implicit. On the surface, it doesn't look like it's asserting anything. Wrapping the equality check in assert { ... } makes it loud and clear that "this is what I'm asserting".
• If this worked, a user might expect additional equality checks (i.e. not just the last line of the example) to work as assertions as well--because, on the surface, it looks like it just magically works to check equality anywhere in the it block. But of course that's not going to work. Alternately, the user would have to do something like:

let(:x) { ... }
let(:y) { ... }
let(:z) { ... }

it "makes two assertions"
  assert { x == y }
  x == z
end

...but this seems weird. The user is essentially nesting an assert in an assert here (since the it block acts like an assert itself). It's inconsistent.

I guess part of the point of this is to strongly encourage "one assertion per example"? My rule of thumb is:

• For unit tests, keep it to one behavior per example. There may or may not be multiple assertions/expectations to express the one behavior. Multiple ones is often a signal that I may be testing multiple behaviors, but often not.
• In end-to-end acceptance tests and integration tests (what I've started calling my tests that focus on one class but communicate with something external...e.g. a model test talking to the DB, or an API wrapper class talking to the 3rd party API), I don't worry about one assertion/expectation per example at all....since test speed is so important to me and these naturally take longer, I tend to make lots of assertions to get more verification value out of the time it takes to run the test.

Anyhow...if I used wrong (which I don't, although, I've often thought it looks really cool...but rspec-expectations has always met my needs well), I would not want this behavior for the reasons I've outlined above.

That said, maybe my concerns are just a problem for my style of specs, and may not be an issue for most of other people. I don't know.

[alexch]

I do think it's interesting!

It turns out that it is only looking at the last statement, but the problem has to do with the scope. We execute first the block, then the last statement and each subexpression, inside the scope of the outside block. So when we get around to evaluating the "x" in "(x == y)" it's in a scope where x doesn't exist yet.

It's probably a bit more complicated than that -- and I have no idea where that "yaml" call is coming from -- but you get the basic idea. Side effects are hard.

On Mon, Jun 18, 2012 at 6:12 AM, David Chelimsky reply@reply.github.com wrote:

So I have this kinda crazy idea for integrating wrong with rspec:

describe User do
 it "has the read-only role by default" do
   User.new.role == Role::READ_ONLY
 end
end

The idea is that, if configured to do so, rspec would forward the block passed to it to assert (or maybe wrong could expose a function to support similar w/o having to include anything). This way you don't even need to fuss over "assert" v "expect" - the block passed to it has to conform to the same rules as a block passed to wrong's assert.

As a preliminary experiment, I tried this:

require 'wrong/adapters/rspec'

describe "something" do
 it "does something" do
   assert do
     x = 5
     y = 3
     x == y
   end
 end

 it "does something else" do
   x = 5
   y = 3
   assert { x == y }
 end
end

When I ran this I got the following output:

FF

Failures:

 1) something does something
    Failure/Error: assert do
      Expected x = 5
      y = 3
      (x == y)
      , but
          (x == y) raises NameError: undefined local variable or method `x' for #<RSpec::Core::ExampleGroup::Nested_1:0x007fe41bdd35e0>
          x raises NameError: undefined local variable or method `x' for #<RSpec::Core::ExampleGroup::Nested_1:0x007fe41bdd35e0>
          y raises NoMethodError: undefined method `yaml' for nil:NilClass
    # ./example_spec.rb:5:in `block (2 levels) in <top (required)>'

 2) something does something else
    Failure/Error: assert { x == y }
      Expected (x == y), but
          x is 5
          y is 3
    # ./example_spec.rb:15:in `block (2 levels) in <top (required)>'

As you can see, wrong does a great job of providing feedback to the example with the single line block, but the feedback for the multi-line block is a bit useless. So the question is: can wrong be modified to only care about the last statement in the block? Of course, the other question is: do you think this idea is interesting enough to try to make it happen?

---

Reply to this email directly or view it on GitHub: https://github.com/sconover/wrong/issues/21

Alex Chaffee - alex@stinky.com http://alexchaffee.com http://twitter.com/alexch