PRX / meta.prx.org

Meta data for all PRX services and apps.
0 stars 0 forks source link

Add metrics basic test #54

Closed pkarman closed 6 years ago

pkarman commented 6 years ago

There is a big TODO in here which really reflects an open question in my mind. These acceptance tests take a long time to run, particular when multiple systems need to interact, so adding pieces that require poll+wait only exacerbates that problem. Curious what others think.

pkarman commented 6 years ago

thanks for the feedback @cavis and @gcampo88 - is this still failing for you locally? Any other obstacles to merging this?

gcampo88 commented 6 years ago

@pkarman afraid I'm still having tests fail locally, on same error for publish-audio + metrics:

publish::js::audio
  test_0001_creates new episode with RSS feed                    ERROR (28.15s)
Capybara::ElementNotFound:         Capybara::ElementNotFound: Unable to find visible button "Saved" that is disabled
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
            /Users/gigiblum-campo/Desktop/PRX_repos/more repos/meta.prx.org/test/support/publish_dsl.rb:187:in `create_series!'
metrics::js::basic
  test_0001_registers when podcast is played                     ERROR (33.76s)
Capybara::ElementNotFound:         Capybara::ElementNotFound: Unable to find visible button "Saved" that is disabled
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:314:in `block in synced_resolve'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/base.rb:85:in `synchronize'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:302:in `synced_resolve'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:37:in `find'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/node/finders.rb:179:in `find_button'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/session.rb:792:in `block (2 levels) in <class:Session>'
            /Users/gigiblum-campo/.rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/capybara-2.15.4/lib/capybara/dsl.rb:50:in `block (2 levels) in <module:DSL>'
            /Users/gigiblum-campo/Desktop/PRX_repos/more repos/meta.prx.org/test/support/publish_dsl.rb:187:in `create_series!'
            /Users/gigiblum-campo/Desktop/PRX_repos/more repos/meta.prx.org/test/metrics/basic_test.rb:34:in `setup_fixtures'

looks like this may be an existing issue though, not necessarily specific to this PR -- it's currently failing out in the wild as reported in #ops-deploys. so, not sure how best to handle-- ?

pkarman commented 6 years ago

@gcampo88 that exception is being thrown because the browser is getting a Got 500 from POST https://cms.staging.prx.tech/api/v1/series/:series_id/distributions error on the page, so the "Save" button never activates. So, it's a legit error (not a problem with the tests per se, but a problem with the env).

pkarman commented 6 years ago

Working for me locally now as well.

pkarman commented 6 years ago

@gcampo88 @cavis - are we ok to merge this now? Or do we have other things to resolve first re: our CI/CD flow and these tests?

cavis commented 6 years ago

I'm good on this.

gcampo88 commented 6 years ago

Ditto

gcampo88 commented 6 years ago

any objections to merging this now?