Open trans opened 10 years ago
With spring, can use spring rake test | tapout pretty
, with guard-minitest,
guard :minitest, spring: true, cli: '| tapout pretty' do
# Rails 4
watch(%r{^app/(.+)\.rb}) { |m| "test/#{m[1]}_test.rb" }
watch(%r{^app/controllers/application_controller\.rb}) { 'test/controllers' }
watch(%r{^app/controllers/(.+)_controller\.rb}) { |m| "test/integration/#{m[1]}_test.rb" }
watch(%r{^app/views/(.+)_mailer/.+}) { |m| "test/mailers/#{m[1]}_mailer_test.rb" }
watch(%r{^lib/(.+)\.rb}) { |m| "test/lib/#{m[1]}_test.rb" }
watch(%r{^test/.+_test\.rb})
watch(%r{^test/test_helper\.rb}) { 'test' }
end
@trans Sorry for my questions, minitap can't work with pry, if use binding.pry
in test file, run ruby -Itest test/models/post_test.rb | tapout
, tests will stop, input exit, got:
ruby -Itest test/models/post_test.rb | tapout
exit
/Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:369:in `parse': (<unknown>): control characters are not allowed at line 1 column 1 (Psych::SyntaxError)
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:369:in `parse_stream'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:317:in `parse'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:244:in `load'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:47:in `handle'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:34:in `<<'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:24:in `consume'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/cli.rb:80:in `cli'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/bin/tapout:3:in `<top (required)>'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/bin/tapout:23:in `load'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/bin/tapout:23:in `<main>'
post_test.rb
require 'test_helper'
class PostTest < ActiveSupport::TestCase
test "the truth" do
@post = Fabricate(:post)
assert @post.valid?
binding.pry
end
end
Take off the | tapout
and look at the raw output. What does it look like?
Btw, what's happening is you're piping pry's interface into tapout, and tapout is trying to interpret that as TAP-Y/J test output. I probably can teach tapout to stop when it hits a document end marker, i.e. ...
. But I am not sure how to release control back to Pry. Maybe tapout would have to fall into a mode where it is doing nothing but routing stdin and stdout. I don't know, that seems odd. Will have to think about it. Any suggestions welcome.
Take off the | tapout, it works,
$ rake test
---
type: suite
start: '2013-11-18 00:23:45'
count: 18
seed: 62546
rev: 4
From: /Users/doabit/Sites/test/rails/tmp/minit/blog/test/models/post_test.rb @ line 7 PostTest#test_the_truth:
4: test "the truth" do
5: @post = Fabricate(:post)
6: assert @post.valid?
=> 7: binding.pry
8: end
$ rake test | tapout
exit
/Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:369:in `parse': (<unknown>): control characters are not allowed at line 1 column 1 (Psych::SyntaxError)
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:369:in `parse_stream'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:317:in `parse'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/psych-2.0.1/lib/psych.rb:244:in `load'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:47:in `handle'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:34:in `<<'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/parsers/yaml.rb:24:in `consume'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/lib/tapout/cli.rb:80:in `cli'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/tapout-0.4.2/bin/tapout:3:in `<top (required)>'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/bin/tapout:23:in `load'
from /Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/bin/tapout:23:in `<main>'
With spring, it couldn't work, either with | tapout
or not. https://gist.github.com/doabit/7515078
Hmm... is spring+pry working when you don't use tap output?
Yes, i also tested minitest-reporters
, it can work with spring + pry. i thought, tapout wait minitap to return yml or json format result, but binding.pry
broken it
It pipes the output as it happens. If it waited then nothing would output until the very end, after all tests were run.
Not sure why it shouldn't work with spring if the output is not being piped to tapout. It's just a standard Minitest reporter, so it should work with anything. But then I am not sure how spring works. Did you say that minitap + spring works okay w/o pry in the mix?
Yes, if not use minitap, everything works well, i also can work with spring + pry + minitest-reporters. I thought, minitap reuturn json or yml fomat result, but when tests stoped, would get JSON::ParserError or (Psych::SyntaxError)
$ rake test | tapout
exit
/Users/doabit/.rbenv/versions/2.0.0-p247/gemsets/4.1.0.beta/gems/json-1.8.1/lib/json/common.rb:155:in
`parse': 795: unexpected token at '{"type"=>"suite", "start"=>"2013-11-18 01:01:50", "count"=>18, "seed"=>56585, "rev"=>4} (JSON::ParserError)
Pry with minitest-reporterts
$ rake test
Started
PostTest
[WARN] cannot load such file -- awesome_print
$ gem install awesome_print
From: /Users/doabit/Sites/test/rails/tmp/minit/blog/test/models/post_test.rb @ line 7 PostTest#test_the_truth:
4: test "the truth" do
5: @post = Fabricate(:post)
6: assert @post.valid?
=> 7: binding.pry
8: end
[1] pry(#<PostTest>)> exit
test_the_truth PASS (2.18s)
PostsControllerTest
test_should_create_post PASS (0.02s)
test_should_destroy_post PASS (0.01s)
test_should_get_edit PASS (0.04s)
test_should_get_index PASS (0.01s)
test_should_get_new PASS (0.05s)
test_should_show_post PASS (0.01s)
test_should_update_post PASS (0.01s)
PostsTest
test_the_truth PASS (0.05s)
UserTest
test_the_truth PASS (0.00s)
UsersControllerTest
test_should_create_user PASS (0.02s)
test_should_destroy_user PASS (0.01s)
test_should_get_edit PASS (0.01s)
test_should_get_index PASS (0.01s)
test_should_get_new PASS (0.01s)
test_should_show_user PASS (0.01s)
test_should_update_user PASS (0.01s)
UsersTest
test_login_player PASS (0.03s)
Finished in 2.48854s
18 tests, 30 assertions, 0 failures, 0 errors, 0 skips
Spring + pry + minitest-reporters
$ spring rake test
Started
PostTest
[WARN] cannot load such file -- awesome_print
$ gem install awesome_print
From: /Users/doabit/Sites/test/rails/tmp/minit/blog/test/models/post_test.rb @ line 7 PostTest#test_the_truth:
4: test "the truth" do
5: @post = Fabricate(:post)
6: assert @post.valid?
=> 7: binding.pry
8: end
[1] pry(#<PostTest>)> exit
test_the_truth PASS (2.42s)
PostsControllerTest
test_should_create_post PASS (0.02s)
test_should_destroy_post PASS (0.01s)
test_should_get_edit PASS (0.09s)
test_should_get_index PASS (0.01s)
test_should_get_new PASS (0.01s)
test_should_show_post PASS (0.01s)
test_should_update_post PASS (0.01s)
PostsTest
test_the_truth PASS (0.05s)
UserTest
test_the_truth PASS (0.00s)
UsersControllerTest
test_should_create_user PASS (0.02s)
test_should_destroy_user PASS (0.01s)
test_should_get_edit PASS (0.01s)
test_should_get_index PASS (0.01s)
test_should_get_new PASS (0.05s)
test_should_show_user PASS (0.01s)
test_should_update_user PASS (0.01s)
UsersTest
test_login_player PASS (0.02s)
Finished in 2.77927s
18 tests, 30 assertions, 0 failures, 0 errors, 0 skips
Yea, it makes perfect sense why Pry doesn't work when piping output to tapout. Unlike minitest-reporters which just integrates itself into minitest so there is only one program running, with minitap we have two different programs running, minitest and tapout, and they are communicating via a pipe --the output from minitest is being passed into tapout, and tapout is interpreting that input as tap-y/j formatted test output. If suddenly some other data comes across the pipe, e.g. a Pry command prompt, tapout cannot interpret that and a YAML or JSON syntax error is going to happen.
I have an idea that should work around this, see https://github.com/rubyworks/tapout/issues/6. But I still have to figure out the details which may take a bit of time.
However none of this explains why spring + minitap + pry doesn't work, just as long as the output is NOT being piped to the tapout command (no | tapout
). It is weird that spring + minitap would work, and pry + minitap would work, but not all three. I will have to dig in and see what I can find.
Got it, thanks for your job.
Just released minitap-0.5.2 (along with tapout-0.4.3). For the most part this should resolve this issue and many related ones. For optimal results one should emit a DLE ASCII code (16.chr
) to tell tapout to pause test result processing, and then send a ETB ASCII code (23.chr
) to tell it to resume. e.g.
test "the truth" do
@post = Fabricate(:post)
assert @post.valid?
STDOUT.puts 16.chr
binding.pry
STDOUT.puts 23.chr
end
I use STDOUT
here and not $stdout
b/c test runners generally cache $stdout
as part of test results. In the future this might need to be addressed more precisely in Minitest's api, but STDOUT
does the trick for now.
Note, this doesn't necessarily address spring + pry + minitap (but one can hope!), as there appears to be something else afoot there. Nonetheless I am going to go ahead can close this issue. We can start new more specific issues for anything that remains.
Thanks, i have tested, but it looks still not work well
$ rake test | tapout
exit
Started w/ Seed: 64378
From: /Users/doabit/Sites/test/rails/tmp/minit/mit/test/models/post_test.rb @ line 6 PostTest#test_the_truth:
4: test "the truth" do
5: STDOUT.puts "..." 0;34# tells tapout to pause processing
=> 6: binding.pry
7: STDOUT.puts "---"
8: assert true
9: end
---
---
type: case
subtype: ''
label: PostTest
level: 0
stdout: "$ gem install awesome_print\n\e[0G"
stderr: |
[WARN] cannot load such file -- awesome_print
---
type: test
subtype: ''
status: pass
label: test_the_truth
time: 1.295685
---
type: final
time: 1.296649
counts:
total: 1
pass: 1
fail: 0
error: 0
omit: 0
todo: 0
...
I thought the main resean is tapout. Now the process is minitap -> tapout, if use binding.pry
, tapout won't accept correct result. Maybe minitest-reporters
's process is the only way
It's just a bug in the yaml parser. It should work if you use 16.chr
and 23.chr
. I don't get the warning about awesome_print though. What does your Rakefile test take look like?
I use rails deault Rakefile, if not use rails
$ ruby -Itest test/main_test.rb | tapout
[1] pry(#<Destroying AR models>)> exit
Started w/ Seed: 3239
From: /Users/doabit/Sites/test/rails/tmp/minit/dd/test/main_test.rb @ line 8 Destroying AR models#test_0001_true test:
5: it "true test" do
6: assert true
7: STDOUT.puts "..." 0;34# tells tapout to pause processing
=> 8: binding.pry
9: STDOUT.puts "---"
10: end
---
---
type: case
subtype: ''
label: Destroying AR models
level: 0
stdout: "$ gem install awesome_print\n\e[0G"
stderr: |
[WARN] cannot load such file -- awesome_print
---
type: test
subtype: ''
status: pass
label: true test
time: 2.988121
---
type: test
subtype: ''
status: fail
label: fail test
exception:
message: Failed assertion, no message given.
class: Minitest::Assertion
file: test/main_test.rb
line: 13
source: assert false
snippet:
- 11: ''
- 12: ' it "fail test" do'
- 13: ' assert false'
- 14: ' end'
- 15: end
backtrace:
- test/main_test.rb:13
time: 2.989053
---
type: final
time: 2.990012
counts:
total: 2
pass: 1
fail: 1
error: 0
omit: 0
todo: 0
...
test_helper.rb
ENV["RACK_ENV"] = "test"
require "minitest"
require "minitest/autorun"
require "minitap"
Minitest.reporter = Minitap::TapY
I have the YAML parser fixed and will release shorty. It turns out the new document marker isn't even needed to restart. But really the more robust solution is to use the codes 16.chr
and 23.chr
. For convenience, in test_helper.rb
one could do something like:
def pry!(binding_of_caller)
STDOUT.puts 16.chr
binding_of_caller.pry
STDOUT.puts 23.chr
end
Then use it:
test "the truth" do
@post = Fabricate(:post)
assert @post.valid?
pry!(binding)
end
Pushed tapout-0.4.4.
I have tested, maybe my problem, it still not work, run
$ ruby -Itest test/main_test.rb | tapout
stoped here:
[1] pry(#<Destroying AR models>)>
Then, input exit
1] pry(#<Destroying AR models>)> exit
From: /Users/doabit/Sites/test/rails/tmp/minit/dd/test/main_test.rb @ line 6 Destroying AR models#test_0001_true test:
5: it "true test" do
=> 6: pry!(binding)
7: assert true
8: end
Started w/ Seed: 15732
.
Finished in 2.693s (0.371 test/s, 2.692543s avg.)
1 tests: 1 pass, 0 fail, 0 errs, 0 todo, 0 omit
I also tried to modify minitap like this
def tapout_before_suite()
result = super().to_json
Tapout::JsonParser.new({format: 'dot', input: StringIO.new(result)})
end
But it not work, i'll try it again, thanks.
That's confusing. Your output almost looks correct except somehow Pry stepped in before the From ...
stuff?
@doabit I really appreciate all your help on this. You've already helped improve minitap/tapout in ways I did not think possible. Thank you.
It's been a bit difficult to compare results b/c we are not working off the same code base. Do you think you can create a public test repo of a mock Rails project that we could both use for this?
Awesome! That is a Huge help.
I've already tracked down one issue that solves spring rake test
. -- It was the arity code, which I expected might be a problem. I wish there was a cleaner way to do it. But I was able to improve it and get it working.
The other issue with | tapout
is most likely a stdout flushing issue. I need to figure out where and how to ensure stdout is not cached.
I thought to use | tapout
maybe a not good idea. The problem is minitap
must return a complete json or yml result, then tapout
converted them and print. But, when we use pry
, it would break the process, minitap
cound't return an compelte result. I have no idea how to solve it, look forward to your solution, thanks
Not so, it doesn't have to return a complete json or yml. It's a data stream, so if everything is working as it should, it processes the stream continuously in discrete chunks, not all at once.
I made some headway in understanding the problem. Have a look at https://github.com/pry/pry/issues/1011
Oh, i see. I always thought tapout
would format result until minitap
completed. eg.
$ rake test | tapout
Started w/ Seed: 64378
From: /Users/doabit/Sites/test/rails/tmp/minit/mit/test/models/post_test.rb @ line 6 PostTest#test_the_truth:
4: test "the truth" do
5: STDOUT.puts "..." 0;34# tells tapout to pause processing
=> 6: binding.pry
7: STDOUT.puts "---"
8: assert true
9: end
---
type: test
subtype: ''
status: pass
label: test_the_truth
time: 1.295685
---
type: final
time: 1.296649
counts:
total: 1
pass: 1
fail: 0
error: 0
omit: 0
todo: 0
...
It didn't format results. but YML.
I am suddenly getting:
Could not find gem 'arel (~> 5.0.0) ruby', which is required by gem 'rails (>= 0) ruby', in any of the sources.
when I run bundle install
on the minitap_demo project.
arel (~> 5.0.0) ? I should arel (4.0.1)
https://github.com/doabit/minitap_demo/blob/master/Gemfile.lock#L60
I had to rebundle. It's okay, I fixed by adding gem 'arel', github: 'rails/arel'
.
Oh, may be rails updated. It works in my computer
Okay. Things are looking up, finally. Turns out it had nothing to do with Pry, Rails or Rake. All that was needed was to add io.sync = true
. I had tried that before --which is why I was thinking it had something to do with one of the afore-mentioned projects. But now I think it was actually Minitest affecting it, b/c all I had to do was make sure I called io.sync = true
after initializing MiniTest's reporter base class, instead of before, and it worked.
Great, i have updated minitap
to 0.5.3, it works with pry now. But there is still a problem, we must use:
def pry!(binding_of_caller)
STDOUT.puts 16.chr
binding_of_caller.pry
STDOUT.puts 23.chr
end
I thought many people may be more like to use binding.pry
directly.
I've been playing with some ways to improve on that, but it's just a matter of how things work. I don't think there is any way to avoid it all together. One idea:
require 'tapout/utitlity'
tapout.pause { binding.pry }
Another option is to override the #pry
method, but that probably has it's own set of problems.
Maybe just offering an alternative:
require 'tapout/utitlity'
binding.tapry # ?
Yes, that is indeed, Add it into README maybe the best way now.
From issue #5:
"I have tried https://github.com/rubyworks/tapout/wiki/Quick-Start, but it can't work well with rails 4, i think to add config like Minitest::Reporters.use! Minitest::Reporters::SpecReporter.new would be great. then it could work with spring or zeus."