AgileVentures / projectscope

MVP dashboard for ProjectScope, using new gems architecture developed by AV folks
2 stars 14 forks source link

builds failing for README changes #46

Closed tansaku closed 8 years ago

tansaku commented 8 years ago

seen this a few times now with errors like:

    And I should see "0.22" within "#WebsiteOne_slack_metric"    # features/step_definitions/web_steps.rb:35
      expected to find text "0.22" in "0.15" (RSpec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/web_steps.rb:108:in `/^(?:|I )should see "([^"]*)"$/'
      ./features/step_definitions/web_steps.rb:36:in `block (2 levels) in <top (required)>'
      ./features/step_definitions/web_steps.rb:29:in `block in with_scope'
      ./features/step_definitions/web_steps.rb:29:in `with_scope'
      ./features/step_definitions/web_steps.rb:36:in `/^(.*) within (.*[^:])$/'
      features/view_metrics_with_secrets.feature:43:in `And I should see "0.22" within "#WebsiteOne_slack_metric"'
Failing Scenarios:
cucumber features/view_metrics_with_secrets.feature:39 # Scenario: update metrics for projects

which is making it look like we have some kind of leak in the VCR sandboxing

tansaku commented 8 years ago

there are no new files or changes being made. the VCR config looks basically correct:

require 'vcr'

VCR.configure do |c|
  c.hook_into :webmock
  c.cassette_library_dir = 'features/support/fixtures/cassettes'
  c.ignore_localhost = true
  c.default_cassette_options = {
      :match_requests_on => [
          :method,
          VCR.request_matchers.uri_without_param(:imp, :prev_imp)
      ],
      # record: :new_episodes # uncomment this line to add new HTTP requests 
  }
  c.filter_sensitive_data("<GITHUB_ACCESS_TOKEN>") { ENV['GITHUB_ACCESS_TOKEN'] }
  c.filter_sensitive_data("<SLACK_ACCESS_TOKEN>") { ENV['SLACK_ACCESS_TOKEN'] }
  c.filter_sensitive_data("<PIVOTAL_TRACKER_API_TOKEN>") { ENV['PIVOTAL_TRACKER_API_TOKEN'] }
end

VCR.cucumber_tags do |t|
  t.tag '@vcr', :use_scenario_name => true
end
tansaku commented 8 years ago

hmm, I wonder if the rake task is running in a separate thread and not being sandboxed by VCR ...

tansaku commented 8 years ago

so the funny thing is we are hitting the rake task in the other cucumber test, and that's stable and fine - hmmm

tansaku commented 8 years ago

so the difference is that the view_metrics_with_secrets.feature has a slack metric using a secret token ... added show me the page ... can't see anything suspicious - feature passes if I change the metric value to match ... hmmm, will do PR on that, but I've done that fix a couple of times already - feels like there's a dependency I'm missing ...

tansaku commented 8 years ago

I wonder if there is something indeterministic inside the slack gem - but tests are all green on that gem ...