search

socketry / nio4r

Cross-platform asynchronous I/O primitives for scalable network clients and servers.
Other
968 stars 86 forks source link

Flakey tests on macOS #184

Closed ioquatix closed 5 years ago

ioquatix commented 6 years ago

For some reason, on my Mac (which is shite for networking anyway) the tests are quite flakey:

Failures:

  1) UDPSocket behaves like an NIO selectable selects readable objects
     Failure/Error: sock.bind("127.0.0.1", udp_port)

     Errno::EADDRINUSE:
       Address already in use - bind(2) for "127.0.0.1" port 23456
     Shared Example Group: "an NIO selectable" called from ./spec/nio/selectables/udp_socket_spec.rb:42
     # ./spec/nio/selectables/udp_socket_spec.rb:10:in `bind'
     # ./spec/nio/selectables/udp_socket_spec.rb:10:in `block (2 levels) in <top (required)>'
     # ./spec/support/selectable_examples.rb:7:in `block (2 levels) in <top (required)>'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:115:in `block in run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `loop'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:33:in `block (2 levels) in setup'

  2) TCPSocket behaves like an NIO selectable selects readable objects
     Failure/Error: expect(ready).to be_an Enumerable
       expected nil to be a kind of Enumerable
     Shared Example Group: "an NIO selectable" called from ./spec/nio/selectables/tcp_socket_spec.rb:69
     # ./spec/support/selectable_examples.rb:9:in `block (2 levels) in <top (required)>'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:115:in `block in run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `loop'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:33:in `block (2 levels) in setup'

  3) TCPSocket behaves like an NIO bidirectional stream keeps readiness after the selectable has been closed
     Failure/Error: expect(m.readiness).to eq(:rw)

       expected: :rw
            got: :w

       (compared using ==)

       Diff:
       @@ -1,2 +1,2 @@
       -:rw
       +:w

     Shared Example Group: "an NIO bidirectional stream" called from ./spec/nio/selectables/tcp_socket_spec.rb:71
     # ./spec/support/selectable_examples.rb:60:in `block (3 levels) in <top (required)>'
     # ./spec/support/selectable_examples.rb:59:in `select'
     # ./spec/support/selectable_examples.rb:59:in `block (2 levels) in <top (required)>'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:115:in `block in run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `loop'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:104:in `run'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
     # /Users/samuel/.rvm/gems/ruby-2.4.0/gems/rspec-retry-0.5.6/lib/rspec/retry.rb:33:in `block (2 levels) in setup'

Finished in 1.24 seconds (files took 0.23694 seconds to load)
104 examples, 3 failures, 1 pending

Failed examples:

rspec './spec/nio/selectables/udp_socket_spec.rb[1:1:1]' # UDPSocket behaves like an NIO selectable selects readable objects
rspec './spec/nio/selectables/tcp_socket_spec.rb[1:1:1]' # TCPSocket behaves like an NIO selectable selects readable objects
rspec './spec/nio/selectables/tcp_socket_spec.rb[1:3:2]' # TCPSocket behaves like an NIO bidirectional stream keeps readiness after the selectable has been closed

Randomized with seed 44170
ioquatix commented 6 years ago

From @tarcieri

@ioquatix the test suite has pretty much always been buggy on OS X, and the failures are both sporadic and very difficult to reproduce in something like a pry session