search

tweetstream / em-twitter

Twitter Streaming API client for EventMachine
http://rubygems.org/gems/em-twitter
MIT License
42 stars 16 forks source link

Test suite fails with Ruby 2.0 #11

Closed auxbuss closed 11 years ago

auxbuss commented 11 years ago

I'm starting to look at the tweetstream ruby2 issue raise here: https://github.com/intridea/tweetstream/issues/117

I installed ruby-2.0.0-p247 (I'm using 195, so this is effectively a clean install), did bundle install, then rake spec.

I'm posting this since it's a failure, rather than a stall, and might be immediately useful.

I ran the test suite three times. The first time:

Finished in 1 minute 7.37 seconds
48 examples, 1 failure

The second time:

Finished in 25.07 seconds
23 examples, 1 failure

The third time the suite stalled.

$ rake spec
...............................................F

Failures:

  1) EM::Twitter::Connection reconnections reconnector setting after successful connect resets the network failure reconnector
     Failure/Error: Unable to find matching line from backtrace
       Exactly one instance should have received the following message(s) but didn't: reset
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-mocks-2.14.1/lib/rspec/mocks/any_instance/recorder.rb:84:in `verify'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-mocks-2.14.1/lib/rspec/mocks/space.rb:18:in `block in verify_all'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-mocks-2.14.1/lib/rspec/mocks/space.rb:17:in `each_value'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-mocks-2.14.1/lib/rspec/mocks/space.rb:17:in `verify_all'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-mocks-2.14.1/lib/rspec/mocks.rb:18:in `verify'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/mocking/with_rspec.rb:18:in `verify_mocks_for_rspec'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example.rb:313:in `verify_mocks'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example.rb:305:in `run_after_each'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example.rb:120:in `block in run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example.rb:254:in `with_around_each_hooks'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example.rb:111:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:400:in `block in run_examples'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:396:in `map'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:396:in `run_examples'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:381:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `block in run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `map'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `block in run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `map'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/example_group.rb:382:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/command_line.rb:28:in `block (2 levels) in run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/command_line.rb:28:in `map'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/command_line.rb:28:in `block in run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/reporter.rb:58:in `report'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/command_line.rb:25:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/runner.rb:80:in `run'
     # /usr/local/rvm/gems/ruby-2.0.0-p247/gems/rspec-core-2.14.1/lib/rspec/core/runner.rb:17:in `block in autorun'

Finished in 1 minute 7.37 seconds
48 examples, 1 failure

Failed examples:

rspec ./spec/em-twitter/connection_reconnect_spec.rb:34 # EM::Twitter::Connection reconnections reconnector setting after successful connect resets the network failure reconnector
auxbuss commented 11 years ago 
$ rake spec
...................................................................F

Failures:

  1) EM::Twitter::Connection reconnections reconnector setting on 4xx responses uses the application failure reconnector
     Failure/Error: expect(client.connection.reconnector).to be_kind_of(Reconnectors::ApplicationFailure)
       expected #<EventMachine::Twitter::Reconnectors::NetworkFailure:0xa0f8250 @reconnect_timeout=0.25, @reconnect_count=0> to be a kind of EventMachine::Twitter::Reconnectors::ApplicationFailure
     # ./spec/em-twitter/connection_reconnect_spec.rb:65:in `block (6 levels) in <top (required)>'
     # ./spec/em-twitter/connection_reconnect_spec.rb:62:in `block (4 levels) in <top (required)>'

Finished in 47.37 seconds
68 examples, 1 failure

Failed examples:

rspec ./spec/em-twitter/connection_reconnect_spec.rb:61 # EM::Twitter::Connection reconnections reconnector setting on 4xx responses uses the application failure reconnector
sferik commented 11 years ago

The reason you’re seeing a different number of examples running each time you run the tests is a result of the RSpec configuration. This file specifies --order random, which runs the suite in random order to prevent order-coupling between specs, and --fail-fast, which aborts the suite as soon as it encounters one failure.

The behavior you’re seeing is as expected.

auxbuss commented 11 years ago

Oh, I was trying to provide data to help solve the stalling tests issue in tweetstream. I thought it would be useful. I wasn't in the least bothered by the random number of tests.

stve commented 11 years ago

These issues are not with the suite itself but with Ruby 2.0. I took the liberty of updating the issue title and am reopening.

@auxbuss thanks for taking a look into this, this is very much in line with my experience running the test suite as well. I've not had a chance to dive into this further yet. I haven't seen that second failure either. No doubt, that's a result of the random ordering of the tests

auxbuss commented 11 years ago

I've isolated the problematic tests.

The stalling spec is connection_error_handling_spec.rb.

The failing spec is connection_reconnect_spec.rb.

When running under rake, note that terminating a stalled connection_error_handling_spec will leave an instance of mockingbird running as a child process of the ruby instance running the test under rpsec.

auxbuss commented 11 years ago

Observation: Inserting a puts at the start and end, and between each of the nine tests in connection_error_handling_spec.rb, plus adding some debug messages into error_callback_invoked() yielded:

$ rspec spec/em-twitter/connection_error_handling_spec.rb 
here 01
here 02
here 03
here 04
here 05
here 06
here 07
here 08
here 09
here 10
*** enter error_callback_invoked()
*** start em block
*** end em block
<test stalls here. error_callback_invoked() never exits>
stve commented 11 years ago

@auxbuss thanks for taking the time to research the issues and narrowing down the failures. I'll take a look in the next day or two to see if I can't get to the bottom of this. I had experienced segfaults under 2.0.0-p0 but it looks like later patch levels are at least letting the suite run which is somewhat encouraging.

auxbuss commented 11 years ago

Since the errors seem to be occurring around em connect/reconnect. I tried running with the pure ruby em (require 'em/pure_ruby'), but the same problems occur. That said, connection_reconnect_spec.rb completes more quickly than the non-pure version. Seems the pure ruby em doesn't go through NetworkFailure at all in the failing test -- it should. Using regular eventmachine does, but it's not clear what it's doing.

auxbuss commented 11 years ago

Something concrete \o/ ruby2 never calls Connection::on_headers_complete(), which I presume is invoked by http_parser.

stve commented 11 years ago

@auxbuss yes, that's exactly what it's supposed to do. Previous usage of http_parser allowed me to simply do Http::Parser.new(self) on the Connection instance. One of the suggested changes by @jfrazee for Ruby 2.0 was implemented for on_body but I didn't change on_headers_complete in a similar manner, see 8e87d7b5f81fe20c43ccb55b93799541054a569c

stve commented 11 years ago

well, Travis seems to be choking on that commit, but i've not had any failures or stalls after 7376820c90fa7bcc417a30bd4892d1f8816d3599. Very encouraging so far.

auxbuss commented 11 years ago

Why does this differ between 1.9 and 2.0?

stve commented 11 years ago

I wish I knew, http_parser.rb has been very stable, and most recent changes appear to be related to JRuby. https://github.com/tmm1/http_parser.rb/commits/master

auxbuss commented 11 years ago

I ran this against 2.0.0-p247, 2.0.0-p195, and 1.9.3-p392. All tests pass.

I can wrap up a gem and test in an app tomorrow (it's 23:00+ here), if that's useful.

stve commented 11 years ago

Closing as tests are now passing with 2.0.