NoRedInk / rspec-retry

retry randomly failing rspec example
MIT License
581 stars 95 forks source link

Issues with Rspec tests for Travis CI randomly failing from time to time #96

Closed koconder closed 3 years ago

koconder commented 6 years ago

Referenced here as well: https://github.com/rspec/rspec/issues/29 / Might be an rspec issue - Not that well versed on ruby and rspec to diagnose.


See example Travis CI build: https://travis-ci.org/duracelltomi/gtm4wp/jobs/380080993 - Randomly it all goes through but sometimes fails for no reason, and would work again on some builds. Example repo where the test code is kept is: https://github.com/koconder/wordpress-test-template/tree/master/spec

In the travis ci build you can see

[Thu May 17 08:02:50 2018] 127.0.0.1:41748 Invalid request (Unexpected EOF)

Relating to:

RSpec::Retry: 2nd try ./baseline.rb:21
RSpec::Retry: 3rd try ./baseline.rb:21
F
RSpec::Retry: 2nd try ./baseline.rb:25
RSpec::Retry: 3rd try ./baseline.rb:25
F
RSpec::Retry: 2nd try ./baseline.rb:30
RSpec::Retry: 3rd try ./baseline.rb:30
F
Cleaning up...
Failures:
  1) WordPress: http://localhost:12000/ -  frontpage Healthy status code 200, 301, 302, 503
     Failure/Error: expect(page).to have_status_of [200,301,302,503]
       expected #<Capybara::Session> to have status of 200, 301, 302, and 503
     # ./baseline.rb:22:in `block (3 levels) in <top (required)>'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:43:in `block (3 levels) in apply'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `times'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `block (2 levels) in apply'
  2) WordPress: http://localhost:12000/ -  frontpage Page includes stylesheets
     Failure/Error: expect(page).to have_css
       expected #<Capybara::Session> to have css
     # ./baseline.rb:26:in `block (3 levels) in <top (required)>'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:43:in `block (3 levels) in apply'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `times'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `block (2 levels) in apply'
  3) WordPress: http://localhost:12000/ -  frontpage Page includes javascript
     Failure/Error: expect(page).to have_js
       expected #<Capybara::Session> to have js
     # ./baseline.rb:31:in `block (3 levels) in <top (required)>'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:43:in `block (3 levels) in apply'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `times'
     # /home/travis/.rvm/gems/ruby-2.4.1/gems/rspec-retry-0.4.0/lib/rspec/retry.rb:34:in `block (2 levels) in apply'
Finished in 6 minutes 41 seconds (files took 0.70182 seconds to load)
3 examples, 3 failures
Failed examples:
rspec ./baseline.rb:21 # WordPress: http://localhost:12000/ -  frontpage Healthy status code 200, 301, 302, 503
rspec ./baseline.rb:25 # WordPress: http://localhost:12000/ -  frontpage Page includes stylesheets
rspec ./baseline.rb:30 # WordPress: http://localhost:12000/ -  frontpage Page includes javascript
The command "cd wp-tests/spec && bundle exec rspec *.rb" exited with 1.

Also not sure if there is a ruby/rvm issue with the:

sh: 1: wp: not found
sh: 1: wp: not found
sh: 1: wp: not found