Use RSpec as default?

[leonelgalan]

RSpec

• Bigger ecosystem [citation needed]
• Tests are more readable (arguably, this could achieve by using Minitest::Spec, but at that point, why not use the "original" spec library)
• Separation of rails_helper and spec_helper: allows to test both Rails and non-Rails code.

  Minitest

• Minitest is the default for Ruby
• "rspec is a testing DSL. minitest is ruby."

[BrandonMathis]

I, personally, am a fan of rspec. Their stubbing makes sense. I prefer the 'should' syntax to the expect and assert. There are several other reasons but I would rather discuss them face to face instead on on github.

I'd like to go with whatever the group decides but if you want a spirited debate I am up for that :smiling_imp:

[leonelgalan]

I'm down for RSpec too. I got to play with RSpec 3 recently and working with it was much easier than the last time I tried Minitest. The "should" syntax is still allowed but the "expect" syntax is preferred and the "should" syntax might be disabled by default in the future.

[iandonovan]

I have, in the past, been frustrated by how the overwhelming majority of Google results for generic test fixes assume the use of Rspec. There's gotta be a reason for that, right? I vote Rspec, even if I've only ever used Minitest. Tests are tests, imo. Make some objects, say they're gonna do something, make sure they did.

[joeyjoejoejr]

I'm a fan of rspec for a few major reasons:

• Built-in mocking/stubbing library is much more powerful
• Test spies
• Community support
• Is easily extended, so you can use just about any mocking library, expectation syntax, etc

[reedlaw]

IMHO rspec is one of the best things ruby has going for itself. Other languages (like Java) didn't have nested testing contexts until very recently, or only by hackish workaround. It makes for very readable code and provides documentation as a bonus. Ruby as a language is flexible almost to a fault, so why not take advantage of that fact to enjoy an expressive DSL for testing. Tests are so boring in other languages.

[elfassy]

I prefer minitest, unfortunately seem to be the only one. The syntax is ruby!

#Rspec
obj.should eq(2) #what does the eq method do?

#Minitest
assert obj == 2, "my comment why"

#----

# RSpec
expect {
  post :create, {:post => valid_attributes}
}.to change(Post, :count).by(1)

# MiniTest
assert_difference 'Post.count', 1 do
  post :create, {:post => valid_attributes}
end

#----

#Rspec
let(:news_item) { NewsItem.new }

#Minitest
def news_item
    @news_item ||= NewsItem.new
end

#----

#Minitest with Mocha gem
Time.stub :now, Time.at(0) do   
    # stub goes away once the block is done
   assert obj_under_test.stale?
end

And with minitest you can still use the describe it blocks

[LNauman]

I was taught using rspec so I'm partial, but that's my vote!

[reedlaw]

@elfassy in all your examples I prefer rspec because it's more human readable. The rspec matchers align with ruby's equality matchers:

a.equal?(b) # object identity - a and b refer to the same object
a.eql?(b) # object equivalence - a and b have the same value
a == b # object equivalence - a and b have the same value with type conversions

Rspec has the same syntax:

a.should equal(b) # passes if a.equal?(b)
a.should eql(b) # passes if a.eql?(b)
a.should == b # passes if a == b

But it also gives you options that read better:

a.should be(b) # passes if a.equal?(b)
a.should eq(b) # passes if a == b

When writing tests, given a choice between two options, one of which is more expressive/readable, I will always choose the more expressive option (all other considerations being equal).

[elfassy]

@reedlaw If I understand correctly you find

a.should equal(b)
a.should eql(b)
a.should == b 
a.should be(b)

more expressive than

assert a.equal?(b)
assert a.eql?(b)
assert a == b

I find the assert more expressive myself. I guess my main problem with Rspec is that it highjacks my objects and add methods to them like should. What if I want

class A
 def should
   "I really would like a should method"
  end
end

assert on the other hand is a method of the test itself. I guess we'll have to agree to disagree :)

[RickCarlino]

Should has been deprecated for a long time.

[RickCarlino]

Here we go. I guess enough people complained about it that they shifted to an 'expect' based syntax that probably uses delegates to the real object.

I am still kind of on the fence. The thing I don't like about Rspec is that the syntax is bloated and a lot of time all that extra syntactic sugar (describe/context blocks etc) just start becoming a distraction from actually writing tests. My vote is still for Minitest/Test::Unit.

[BrandonMathis]

One of my biggest issues with minitest is that I cannot run a single test like I can with rspec

ie: rspec spec/lib/class_spec.rb:12

How can I do that with minitest?

[RickCarlino]

There's a plugin called "m". I really like it. It lets you do stuff like m test/models/user:23 like rspec does.

[joeyjoejoejr]

Here's another reason I vote for rspec.

RSpec.configure do |config|
  config.expect_with :test_unit
end

 it "is equal to [1]" do
    assert [1] ==  [1], "expected [1] to equal [1]"
  end

Just that easy, if you like to write tests that way.

[joeyjoejoejr]

Or both.

RSpec.configure do |config|
  config.expect_with :rspec, :test_unit
end

[joeyjoejoejr]

There's also diffing: https://www.relishapp.com/rspec/rspec-expectations/docs/diffing#diff-for-a-multiline-string

One of my biggest problem with assertions is not being able to see why the two things didn't match, include, equal, or whatever.

[elfassy]

From @leonelgalan initial posts, the way i see it is:

RSpec

• Bigger ecosystem
• Larger user base
• Community support
• DSL (in other words, bigger learning curve for people not familiar with it)

I disagree with

• Tests are more readable
  • You can use the describe it syntax with minitest, and assert is more descriptive to me (as mentioned above)
• Separation of rails_helper and spec_helper: allows to test both Rails and non-Rails code.
  • You can name your helper whatever you want (as long as you require it)
• Built-in mocking/stubbing library is much more powerful
  • Mocha gem for minitest is great
• Test spies
  • https://www.relishapp.com/mortice/matahari/docs/spying-in-testunit-minitest

Minitest

• Minitest is the default for Ruby 2.0+
• Tests are written in plain Ruby
• Need to add other gems to make it as complete as Rspec (this could be good since it can lead to more options, but it's also more configuration)
  • minitest-reporters (output coloring/layout)
  • minitest-metadata (for test options like :js => true)
  • mocha (mocking/stubbing)
  • https://github.com/rubyworks/minitap (like reporters but haven't tried it)
  • ...

[BrandonMathis]

I cannot get m to work. It also hasn't been updated since April of 2013.

The inability to run a single test in a file is a big deal-breaker for minitest in my eyes. :(

[elfassy]

@joeyjoejoejr "One of my biggest problem with assertions is not being able to see why the two things didn't match, include, equal, or whatever."

there are two options for you

# Option1: add a custom message (recommended!)
assert a > b, "Gummies should be better than carrots"
assert a =~ /^bear/, "Gummies should start with the text 'Bear'"

#Option 2: Use specific assert commands
assert_empty @labels
assert_equal 2, @traits.size
assert_in_delta @traits.size, 1,1
assert_in_epsilon @traits.size, 1, 1
assert_includes @traits, "skinny jeans"
assert_instance_of Hipster, @hipster
assert_kind_of Enumerable, @labels
assert_match @traits.first, /silly/
assert_nil @labels.first
assert_operator @labels.size, :== , 0
assert_output("Size: 2") { print "Size: #{@traits.size}"}
assert_raises(NoMethodError) { @traits.foo }
assert_respond_to @traits, :count
assert_same @traits, @traits, "It's the same object silly"
assert_send [@traits, :values_at, 0]
assert_silent { "no stdout or stderr" }
assert_throws(Exception,'is empty') {throw Exception if @traits.any?}

Either options will return a nice message explaining why the test failed

[elfassy]

@BrandonMathis see this answer http://stackoverflow.com/a/9310490/454375

ruby my_test -n test_my_test

[joeyjoejoejr]

@BrandonMathis, I've found that I've had the same problem with the minitest ecosystem.

@elfassy, i guess my question now is that since it's so easy to use the assertions you like in rspec, is there any other complaint about it?

Also I'm not sure how having to know to 17 assert methods is any different that knowing 17 expectation matchers. I don't see expect(@traits).to equal(@trails) being any less ruby than assert_same @traits, @traits

[leonelgalan]

Definitely option two, just like rspec, option one is the equivalent to: expect(a > b).to be true and this is not good.

Syntax preferences apart, and I understand how some might prefer the Test::Unit syntax. I would never suggest to use one inside the other (spec syntax with Minitest or asset syntax in Rspec)

[joeyjoejoejr]

Why not use asserts in rspec? They explicitly show it as an option in their docs? Also minitest has built in spec syntax.

[leonelgalan]

@joeyjoejoejr, because it's unintuitive, for RSpec the preferred way is the expect syntax, should syntax is a close second (a favorite of many), using asserts it's just trying to hard to write tests the Test::Unit way. At that point I beleive you shouldn't be using RSpec.

I'm with Joe, with the "minitest is ruby" argument. DSLs are a big thing in Ruby and they are still Ruby. If you have a DSL for tests, chances are that it's gonna read better for tests. In my opinion it does, but that's a personal preference at the end.

[elfassy]

@leonelgalan why is it no good? As long as you put a message, i don't see what's wrong.

assert @labels.first.nil?, "Labels should have all been removed after deletion of related object"
# Failed: Labels should have all been removed after deletion of related object 

#vs
assert_nil @labels.first
# Failed: Expected nil got <Label>

I agree that it comes down to personal preference. I think we should have everyone's argument and then vote on what most people would rather use. The danger is obviously people voting without knowing about the alternative.

[leonelgalan]

For me it's very clear why, looking at the code example you posted. I guess my issues are:

• Lack of specificity
• Having that long message in there, it's like putting a comment because your code is not clear enough.

---

On how are we going to choose, I agree with you. I love the conversation this whole thread! I wouldn't mind going with Minitest as long as the ecosystem is as good as RSpec (that's the deal breaker for me). I didn't know about some of the tools mentioned here, specially for Minitest, but I still believe their are more and better tools/documentation for RSpec out there. Just like the TodoApp I would love to see some side by side examples of everything we've talk here before we make a decision.

[iandonovan]

Popping my head in to second @elfassy's assertion (har har har) that Mocha rocks. It's a great mocking & stubbing framework.

[reedlaw]

@elfassy I don't really care whether the syntax prefers assert, should, or expect. My main preference is for the Rspec-style specification over the Minitest-style. Thus, Minitest::Spec is a good second-choice option if Rspec is not chosen. From Minitest's README:

Given that you'd like to test the following class:

  class Meme
    def i_can_has_cheezburger?
      "OHAI!"
    end

    def will_it_blend?
      "YES!"
    end
  end

=== Unit tests

Define your tests as methods beginning with `test_`.

  require "minitest/autorun"

  class TestMeme < Minitest::Test
    def setup
      @meme = Meme.new
    end

    def test_that_kitty_can_eat
      assert_equal "OHAI!", @meme.i_can_has_cheezburger?
    end

    def test_that_it_will_not_blend
      refute_match /^no/i, @meme.will_it_blend?
    end

    def test_that_will_be_skipped
      skip "test this later"
    end
  end

=== Specs

  require "minitest/autorun"

  describe Meme do
    before do
      @meme = Meme.new
    end

    describe "when asked about cheeseburgers" do
      it "must respond positively" do
        @meme.i_can_has_cheezburger?.must_equal "OHAI!"
      end
    end

    describe "when asked about blending possibilities" do
      it "won't say no" do
        @meme.will_it_blend?.wont_match /^no/i
      end
    end
  end

The Minitest::Test version looks like any other testing framework for any other language. I write the same types of tests in C++ with googletest, i.e. write a test class and then one method per test. You say the syntax is ruby, but ruby is a dynamic, flexible language capable of so much more than simple classes with methods. If you want to harness that power (which came at great cost in performance) where else to do so than in tests? If we don't care about the parts of Ruby that are capable of leveraging custom DSLs, why don't we switch to C++ and get a huge performance boost? Tests should make full use of an expressive, readable DSL in order to clearly convey their purpose as well as document the code.

The reason I prefer Rspec over Minitest::Spec is Rspec is more powerful out-of-the-box and I would probably end up adding as many gems as it would take to make Minitest function like Rspec anyway. Similar to the way we tend to choose Rails over Sinatra. Sinatra is lighter, but you're probably going to need all that Rails provides in the long run.

[elfassy]

what do you guys think of http://codegram.github.io/spinach/ (https://github.com/codegram/spinach) Example from the GitLab project: https://github.com/gitlabhq/gitlabhq/tree/master/features

[reedlaw]

@elfassy is Spinach just Cucumber, but better?

[elfassy]

It has the same text parser as cucumber, but it doesn't try to guess values from the text, rather it generates a skeleton. I think it's useful because it forces us to write out all requirements (this could be potentially done by a non dev).

[elfassy]

Example: 1) you write the features https://github.com/gitlabhq/gitlabhq/blob/master/features/admin/groups.feature

2) you fill in the steps https://github.com/gitlabhq/gitlabhq/blob/master/features/steps/admin/groups.rb

[elfassy]

more info here: http://codegram.github.io/spinach-presentation/#1

[BrandonMathis]

IMO: I really like integration tests. They are extremely useful when it comes to reinforcing the stability of an app and most projects that I have written integration tests in have benefited in the long run.

What I hate is integration/feature tests written in straight rspec. They are often messy and super difficult to read.

Tools like Cucumber and Spinach not only make your integration tests easier to follow, they also document your use cases for future developers who may come to the projects. I always jump for job when I see a cucumber directory in a project that I inherit.

[reedlaw]

:+1: looks good to me! I like explicit user stories but I didn't like the pain and slowness of Cucumber. The biggest hurdle though is to get the product owners involved in defining stories and keeping them up to date.

[BrandonMathis]

I don't think that the product owner should be writing cucumber files. That is the job of the developers and it is an excellent tool to use as a form of communication with the PM's but clients should see none of this.

At the end of the day, these feature files are more useful for clarifying the expectations of use-cases internally. They can also be exceptionally useful when onboarding developers.

[reedlaw]

@BrandonMathis I didn't mean they should be responsible for writing them, but product owners should be aware of/responsible for their content. That way they become a sort of requirements document. Otherwise, there would be little benefit to using Gherkin-style syntax to define tests as they would just be an intermediary step between the actual tests. As I understand them, cucumber features exist to bridge the gap between non-developer product owners and back-end developers writing the actual tests.

[BrandonMathis]

I disagree that project owners should be even looking at feature files. Every time I have attempted that, clients have immediately balked at me and asked why I am showing them "code" or they get too deep into the trenches of the details about how a use case is implemented.

Spinach and it's Gherkin-style syntax is best for documenting your use-case. It can go a long way to help a team of developers, PM's, and QA engineers better understand use-cases and their many edge cases.

[joeyjoejoejr]

I agree. I feel that it's most valuable to PMs and QA who may not be fluent in ruby, but aren't afraid of some high level code. I think it gives the PMs a way to verify that the features are complete and a good starting place for QA so they don't duplicate the effort of testing the happy path that is verified to be working. I can't think of a client that I've worked with who would be interested in reading feature files.

On Mon, Aug 18, 2014 at 9:28 AM, BrandonMathis notifications@github.com wrote:

I disagree that project owners should be even looking at feature files. Every time I have attempted that, clients have immediately balked at me and asked why I am showing them "code" or they get too deep into the trenches of the details about how a use case is implemented.

Spinach and it's Gherkin-style syntax is best for documenting your use-case. It can go a long way to help a team of developers, PM's, and QA engineers better understand use-cases and their many edge cases.

— Reply to this email directly or view it on GitHub https://github.com/smashingboxes/box_cutter/issues/2#issuecomment-52491344 .

[reedlaw]

Please read A Case Against Cucumber. Cucumber adds a layer of inderection to tests. The conclusion, "Quit writing cukes unless you can honestly say that there is someone reading them who would not understand pure Ruby," is hard to refute. Why write Cucumber tests unless someone who doesn't know Ruby is part of the process? I don't advocate throwing out Gherkin-style tests. But I would if there is no product owner involved. There is little point to writing tests in plain English and then writing a regex parser to run them if the only ones looking at the tests understand computer language.

[reedlaw]

To clarify my position regarding my discussion with @elfassy about preferring RSpec syntax, I want to say that my priority for testing is readability, but not at all costs. RSpec syntax leverages Ruby features to create an expressive way to write tests. Cucumber, on the other hand, requires an additional step to interpret. That doesn't mean it's without value, just that there better be some good reasons to chose it over more direct acceptance tests (the chief reason being product owner involvement). I also like FitNesse tests, even though they require the creation of an entire set of test fixtures that interpret the tables in the wiki. The justification for all that extra effort is that it gives clients an even more powerful tool for writing their own tests using spreadsheet-like tables. If a client can actually get onboard with that kind of testing, then the feedback you get is invaluable. I'm for simple tests, but I'm also for whatever it takes to get clients to elicit enough detail that you can build a system and be confident that it aligns with the client's expectations. That is, after all, the ultimate goal when working for clients.