Intermittent js test failures

cavis commented 6 years ago

Just capturing places the tests tend to fail.

cavis commented 6 years ago

We only keep 2 weeks of logs currently ... so grabbing everything up until now.

2018/01/09 00:18:21

publish::js::audio
test_0001_creates new episode with RSS feed ERROR (113.31s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src434717001/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src434717001/src/test/publish/basic_test.rb:31:in `block (3 levels) in <top (required)>'
metrics::js::basic
test_0001_registers when podcast is played ERROR (120.39s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src434717001/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src434717001/src/test/metrics/basic_test.rb:35:in `setup_fixtures'
  /codebuild/output/src434717001/src/test/metrics/basic_test.rb:12:in `block (3 levels) in <top (required)>'

2018/01/08 18:46:14

metrics::js::basic
test_0001_registers when podcast is played ERROR (123.09s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src846963045/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src846963045/src/test/metrics/basic_test.rb:35:in `setup_fixtures'
  /codebuild/output/src846963045/src/test/metrics/basic_test.rb:12:in `block (3 levels) in <top (required)>'
publish::js::audio
test_0001_creates new episode with RSS feed ERROR (123.82s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src846963045/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src846963045/src/test/publish/basic_test.rb:31:in `block (3 levels) in <top (required)>'

2018/01/08 18:39:22

metrics::js::basic
test_0001_registers when podcast is played ERROR (132.03s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src307890468/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src307890468/src/test/metrics/basic_test.rb:35:in `setup_fixtures'
  /codebuild/output/src307890468/src/test/metrics/basic_test.rb:12:in `block (3 levels) in <top (required)>'

2018/01/05 18:12:44

publish::js::audio
test_0001_creates new episode with RSS feed ERROR (43.36s)
  Capybara::ElementNotFound: Capybara::ElementNotFound: Unable to find visible button "Publish"
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
  /codebuild/output/src175020877/src/test/support/publish_dsl.rb:263:in `publish_wait_for_publish_button'
  /codebuild/output/src175020877/src/test/support/publish_dsl.rb:303:in `create_episode!'
  /codebuild/output/src175020877/src/test/publish/basic_test.rb:29:in `block (3 levels) in <top (required)>'

2018/01/05 16:55:00

metrics::js::basic
test_0001_registers when podcast is played ERROR (43.84s)
  Capybara::ElementNotFound: Capybara::ElementNotFound: Unable to find visible button "Publish"
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
  /codebuild/output/src259203680/src/test/support/publish_dsl.rb:263:in `publish_wait_for_publish_button'
  /codebuild/output/src259203680/src/test/support/publish_dsl.rb:303:in `create_episode!'
  /codebuild/output/src259203680/src/test/metrics/basic_test.rb:34:in `setup_fixtures'
  /codebuild/output/src259203680/src/test/metrics/basic_test.rb:12:in `block (3 levels) in <top (required)>'

2018/01/05 18:12:44

publish::js::audio
test_0001_creates new episode with RSS feed ERROR (43.36s)
  Capybara::ElementNotFound: Capybara::ElementNotFound: Unable to find visible button "Publish"
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
  /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
  /codebuild/output/src175020877/src/test/support/publish_dsl.rb:263:in `publish_wait_for_publish_button'
  /codebuild/output/src175020877/src/test/support/publish_dsl.rb:303:in `create_episode!'
  /codebuild/output/src175020877/src/test/publish/basic_test.rb:29:in `block (3 levels) in <top (required)>'

2018/01/05 16:55:00

metrics::js::basic
test_0001_registers when podcast is played ERROR (43.84s)
 Capybara::ElementNotFound: Capybara::ElementNotFound: Unable to find visible button "Publish"
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
 /usr/local/bundle/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
 /codebuild/output/src259203680/src/test/support/publish_dsl.rb:263:in `publish_wait_for_publish_button'
 /codebuild/output/src259203680/src/test/support/publish_dsl.rb:303:in `create_episode!'
 /codebuild/output/src259203680/src/test/metrics/basic_test.rb:34:in `setup_fixtures'
 /codebuild/output/src259203680/src/test/metrics/basic_test.rb:12:in `block (3 levels) in <top (required)>'

pkarman commented 6 years ago

@cavis I don't have access to AWS - any chance you could copy/paste the actual error logs here? or is what you've got already here the only thing in the logs?

cavis commented 6 years ago

2018/01/10 17:50:24

publish::js::audio
test_0001_creates new episode with RSS feed ERROR (129.03s)
  NoMethodError: NoMethodError: undefined method `attribute' for nil:NilClass
  /usr/local/bundle/gems/nokogiri-1.8.1/lib/nokogiri/xml/node_set.rb:167:in `attr'
  /codebuild/output/src491727864/src/test/support/publish_dsl.rb:243:in `publish_fetch_rss_media_url'
  /codebuild/output/src491727864/src/test/publish/basic_test.rb:31:in `block (3 levels) in <top (required)>'

cavis commented 6 years ago

@pkarman updated with stack traces inline.

cavis commented 6 years ago

Here's a different one:

2018/01/11 17:12:25

metrics::js::basic
test_0001_registers when podcast is played FAIL (64.03s)
  Minitest::Assertion: expected to find text "Test Episode f9314fe4fce51bf434cb" in "PRX Test Series f9314fe4fce51bf434cb test test Please wait... Dec 31 Jan 2 Jan 4 Jan 6 Jan 8 Jan 10 Episode All Episodes Release Date Total for Period Dec 31 Jan 1 Jan 2 Jan 3 Jan 4 Jan 5 Jan 6 Jan 7 Jan 8 Jan 9 Jan 10 Jan 11 0 0 0 0 0 0 0 0 0 0 0 0 0 < 1 > You're seeing a beta preview of prx.org. Let us know what you think. Use Old Version Company About Us Blog Team Press Funding Donate Support Help Status Terms Privacy Projects Radiotopia Matter Ventures".
  /usr/local/bundle/gems/capybara_minitest_spec-1.0.6/lib/capybara_minitest_spec.rb:42:in `block in define_assertion'
  /codebuild/output/src206953507/src/test/metrics/basic_test.rb:17:in `block (3 levels) in <top (required)>'

sandikbarr commented 6 years ago

This seems like a new one, but still looks like a timing issue.


dovetail-programs::criminal
test_0001_matches the production mp3 FAIL (0.00s)
Minitest::Assertion: Expected: 200
Actual: 404
/codebuild/output/src890411166/src/test/dovetail/programs_test.rb:58:in `block (4 levels) in <top (required)>'

cavis commented 6 years ago

Yeah, I'm kind of confused by that dovetail one - was unable to duplicate locally. Could have been a few different issues - opened a new ticket #73 to cover those.

pkarman commented 6 years ago

Since we have disabled/trimmed the expensive JS tests, let's close this particular catch-all ticket now and open failure-specific ones like #73

PRX / meta.prx.org

Intermittent js test failures #66