Support custom rake tasks

[cameel]

I'm using Apotomo and Cells in my application. Those two provide test:widgets and test:cells rake tasks for running tests. I also have test:lib task that runs tests for generic code that I usually put into lib/ directory. It would be nice if I could run those from tconsole and if all was taking them into account.

In my Rakefile I have this bit that makes rake test run everything:

Rake::Task[:test].enhance { Rake::Task["test:lib"].invoke() }
Rake::Task[:test].enhance { Rake::Task["test:cells"].invoke() }
Rake::Task[:test].enhance { Rake::Task["test:widgets"].invoke() }

Not perfect (does not run the additional tests when rake test fails) but does the job fine. I'm not sure how I could achieve this in tconsole.

I see three ways of implementing this (fom the UI standpoint): 1) Allow one to define names of custom tasks in the config 2) When you give a directory, execute all tests (ending in _test.rb) in that directory. Then test would run all tests, test/widgets would run all tests for widgets (located in test/widgets), etc. 3) Add rake command to run arbitrary rake task from within tconsole

[gma]

1) is already an option; you can extend the default file sets. If you search the README for file_sets and merge a hash into it that defines your extra sets, from what I can tell you should be good to go. Admittedly that isn't working for me, but I suspect I have something weird going on in my project as mocha doesn't load properly either.

2) I'd love to see it work like that. What's surprising is that it tconsole pretends that it's going to run those tests for you, and then comes back after a few seconds having run nothing.

[nalanj]

Hey @cameel. @gma is right, there's already support for custom file sets. You can set up a .tconsole file with something like this in it and it should take care of things for you:

TConsole::Config.run do |config|
   config.file_sets["all"] << "test/cells/**/*_test.rb"
   config.file_sets["all"] << "test/widgets/**/*_test.rb"

   config.file_sets["cells"] = ["test/cells/**/*_test.rb"]
   config.file_sets["widgets"] = ["test/widgets/**/*_test.rb"]
end

I originally tried to base everything off of the Rails rake environment, but some really crazy issues arose from how the Rails rake tasks work, and beyond that I work on a ton of non-Rails projects, so I wanted to have something that would work with or without Rails. I also originally worked exclusively with paths (in TConsole 1.0 you could have run cells/*/ and you would run all the tests in cells), but I decided against continuing down that path because it requires really strong conventions and a lot of trickery to make the more advanced features of TConsole like re-running failed tests work the way they should.

Let me know if you run into problems with that .tconsole file and we can troubleshoot it a bit more.

[cameel]

@gma, @commondream: Thanks for your help. File sets work great for me.

I have actually started my config before I've seen @commondream's answer and did it a little bit more DRY (though with the same end result). Here it is if anyone's interested:

TConsole::Config.run do |config|
  custom_sets = {
    "widgets" => ["test/widgets/**/*_test.rb"],
    "cells"   => ["test/cells/**/*_test.rb"],
    "lib"     => ["test/lib/**/*_test.rb"]
  }

  custom_sets.each do |name, patterns|
    config.file_sets[name] = patterns
    config.file_sets["all"].concat(patterns)
  end
end

As for executing whole directories ad-hoc from the command line: you already allow executing tests from a single file. Doesn't that also interfere with re-running failed tests? Now that I can use file sets it's not critical for me but I think it would still be useful if it could be done without too much effort.

[nalanj]

I actually don't support running tests from a single file anymore - it runs against class names now instead of against filenames. I may, though, add a syntax at some point in the future to add files back in.

[cameel]

Oh, I did not notice that. I have just updated to 1.1.2 and used that feature extensively in earlier versions. Ability to re-run failed tests alleviates this to some extent though.

Still, I see two advantages to the file-based approach: 4) It had tab-completion (though it can probably be done for classes too) 5) I am using coulda for BDD scenarios without Cucumber and it hides class names from the user. It probably defines them under the hood and I'm not sure if they even have names. A test scenario looks like this:

Feature "A" do
  in_order_to "a"
  as_a "b"
  i_want_to "c"

  Scenario "A.1" do
    Given "x" do
      ...
    end

    And "y" do
      ...
    end

    When "z" do
      ...
    end

    Then "w" do
      ...
    end
  end
end

It's not a big deal but the ability to re-run a single scenario was very helpful at times.

[nalanj]

@cameel What does the output look like when you run with Coulda?

We do have tab completion on class names, so that's taken care of for you. It also completes on method names - you just use ClassName#method_name format for method names.

[cameel]

Adding_goal_participants_on_edit_page 2
 E test_adding_a_participant_as_both_coordinator_and_non_coordinator 22.035439s 2-1
 E test_adding_a_participant_that_s_already_on_the_list 5.158013s 2-2
 E test_adding_user_to_the_coordinator_list 4.944794s 2-3
 E test_adding_user_to_the_non_coordinator_list 4.889726s 2-4

Right. The class name is right there (Adding_goal_participants_on_edit_page). And the completion works.

So that solves all my problems for now. Thanks! You made Test::Unit awesome again :)

[nalanj]

:grin: Awesome. If you'll notice the funny little numbers on the left side too (2, 2-1, 2-2, etc.) - those are shorthand, so you can just type in the number if you'd like to run a particular test. That helps speed things up a good bit too.