search

mattheworiordan / capybara-screenshot

Automatically save screen shots when a Capybara scenario fails
MIT License
1.02k stars 168 forks source link

Screenshot in Rspec breaks model/controller tests when used with webmock #90

Open jskirst opened 10 years ago

jskirst commented 10 years ago

Screenshot seems to run on all examples, regardless of type. Normally this isn't an issue, as it doesn't produce a screenshot if Capybara isn't running. But in a model spec, it still tries, which causes Capybara to fire a request to the test server. Webmock will block this by default, as the request has not been stubbed for the example, causing the test to fail as in the example below.

Real HTTP connections are disabled. 
Unregistered request: GET http://127.0.0.1:7777/__identify__ with headers 
{'Accept'=>'*/*', 'Accept-
Encoding'=>'gzip;q=1.0,deflate;q=0.6,identity;q=0.3', 'User-Agent'=>'Ruby'}

Setting Capybara::Screenshot.autosave_on_failure = false in config.before(:each) does not help since it is checked after if Capybara.page.respond_to?(:save_page), which I assume is what fires off the identify request.

Am I missing a config option here? If none already exists, I think the lowest impact way to fix this would be to add a configuration that allows you to completely disable screenshot for certain test types, or move the Capybara::Screenshot.autosave_on_failure check above the Capybara.page.respond_to?(:save_page) check so that it never launches Capybara unless it needs to.

Let me know if I'm missing something obvious. Otherwise happy to create a pull request.

mattheworiordan commented 10 years ago

If you look at https://github.com/mattheworiordan/capybara-screenshot/blob/master/lib/capybara-screenshot/rspec.rb#L48, you will see that we don't tell Capybara to fire off any request, however as you say, we are checking the current_url of the page. If you have not invoked Capybara in a model test which is typical unless you have specified your test is a feature, then I have never experienced any issues with this. In fact, if you look at the model test in our demo app, you will see that Capybara never fires. We could add an option to completely disable Capybara Screenshot, but we've never needed it before. I am not sure why Webmock is getting involved when simply accessing Capybara.page

mattheworiordan commented 9 years ago

@jskirst do you still have this issue? I pushed a large update recently so we are now on 1.* release. It would be good to know if this issue still affects you.

afdev82 commented 6 years ago

I get the same problem too.

This is my stack trace:

 # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/webmock-3.4.2/lib/webmock/http_lib_adapters/net_http.rb:114:in `request'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/2.5.0/net/http.rb:1213:in `get'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara/server.rb:82:in `block in responsive?'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/webmock-3.4.2/lib/webmock/http_lib_adapters/net_http.rb:123:in `start_without_connect'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/webmock-3.4.2/lib/webmock/http_lib_adapters/net_http.rb:150:in `start'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/2.5.0/net/http.rb:609:in `start'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara/server.rb:82:in `responsive?'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara/server.rb:98:in `boot'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara/session.rb:88:in `initialize'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara.rb:304:in `new'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara.rb:304:in `current_session'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara/dsl.rb:45:in `page'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-screenshot-1.0.21/lib/capybara-screenshot/rspec.rb:56:in `block in after_failed_example'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-2.18.0/lib/capybara.rb:351:in `using_session'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-screenshot-1.0.21/lib/capybara-screenshot/rspec.rb:55:in `after_failed_example'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/capybara-screenshot-1.0.21/lib/capybara-screenshot/rspec.rb:97:in `block (2 levels) in <top (required)>'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:357:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:509:in `block in run_owned_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:508:in `each'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:508:in `run_owned_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:595:in `block in run_example_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:594:in `each'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:594:in `run_example_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:465:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:507:in `run_after_example'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:273:in `block in run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:500:in `block in with_around_and_singleton_context_hooks'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:457:in `block in with_around_example_hooks'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:466:in `block in run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:606:in `block in run_around_example_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:342:in `call'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-rails-3.7.2/lib/rspec/rails/adapters.rb:127:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:447:in `instance_exec'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:375:in `execute_with'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:608:in `block (2 levels) in run_around_example_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:342:in `call'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:609:in `run_around_example_hooks_for'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/hooks.rb:466:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:457:in `with_around_example_hooks'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:500:in `with_around_and_singleton_context_hooks'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example.rb:251:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:628:in `block in run_examples'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:624:in `map'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:624:in `run_examples'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:590:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `block (3 levels) in run_specs'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `map'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:118:in `block (2 levels) in run_specs'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/configuration.rb:1926:in `with_suite_hooks'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:113:in `block in run_specs'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/reporter.rb:79:in `report'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:112:in `run_specs'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:87:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:71:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/lib/rspec/core/runner.rb:45:in `invoke'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/rspec-core-3.7.1/exe/rspec:4:in `<top (required)>'
          # /home/antonio/.rbenv/versions/2.5.1/bin/rspec:23:in `load'
          # /home/antonio/.rbenv/versions/2.5.1/bin/rspec:23:in `<top (required)>'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:74:in `load'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:28:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli.rb:424:in `exec'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli.rb:27:in `dispatch'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/cli.rb:18:in `start'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/exe/bundle:30:in `block in <top (required)>'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
          # /home/antonio/.rbenv/versions/2.5.1/lib/ruby/gems/2.5.0/gems/bundler-1.16.2/exe/bundle:22:in `<top (required)>'
          # /home/antonio/.rbenv/versions/2.5.1/bin/bundle:23:in `load'
          # /home/antonio/.rbenv/versions/2.5.1/bin/bundle:23:in `<main>'
          # 
          #   Showing full backtrace because every line was filtered out.
          #   See docs for RSpec::Configuration#backtrace_exclusion_patterns and
          #   RSpec::Configuration#backtrace_inclusion_patterns for more information.

If I disable Webmock, it works.

mattheworiordan commented 6 years ago

Sorry to hear this @afdev82. I'm really bogged down with my day job at present so not sure I will have time to look at this any time soon. I'd definitely welcome a PR if you have the time?

mattheworiordan commented 6 years ago

@afdev82 @jskirst if either of you can create a reproducible example that I can use, I can find some time somehow, on a weekend, to look into this and see if I can find a fix.

afdev82 commented 6 years ago

Hi, yes I cannot post my source code... I'm new to rspec / capybara so I don't know yet how to reproduce it with a sample. I'm currently using another approach to test, I didn't need capybara in the end.

Anyway, this is my Gemfile if this could help:

source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

ruby File.read('.ruby-version').strip

# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 5.1.6'
# Use postgresql as the database for Active Record
gem 'pg', '>= 0.18', '< 2.0'
# Use Puma as the app server
gem 'puma', '~> 3.7'
# Use SCSS for stylesheets
gem 'sassc-rails'
# Use Uglifier as compressor for JavaScript assets
gem 'uglifier', '>= 1.3.0'
# See https://github.com/rails/execjs#readme for more supported runtimes
gem 'mini_racer'

# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.5'
# Use Redis adapter to run Action Cable in production
# gem 'redis', '~> 3.0'
# Use ActiveModel has_secure_password
# gem 'bcrypt', '~> 3.1.7'

gem 'dalli'

# Alchemy CMS
gem 'alchemy-richmedia-essences',
    github: 'adnotam/alchemy-richmedia-essences',
    branch: 'add_video_poster'
gem 'alchemy_cms',
    github: 'adnotam/alchemy_cms',
    branch: 'custom'

# Alchemy - Solidus integration
gem 'alchemy-solidus'

# Solidus
gem 'solidus',
    github: 'adnotam/solidus',
    branch: 'custom'

# Solidus extensions
gem 'solidus_catalog_mode'
gem 'solidus_editor',
    github: 'solidusio-contrib/solidus_editor',
    ref: '9582a64091ea4559387970aab0635f18cbc5726b'
gem 'solidus_i18n',
    github: 'adnotam/solidus_i18n',
    branch: 'custom'
gem 'solidus_multi_domain',
    github: 'solidusio/solidus_multi_domain',
    ref: 'f65fafbc0f2e9aecbc490d670ffdd021e72e8c81'
gem 'solidus_paypal_braintree',
    github: 'adnotam/solidus_paypal_braintree',
    branch: 'custom'
gem 'solidus_paypal_express',
    github: 'adnotam/solidus_paypal_express'
gem 'solidus_print_invoice',
    github: 'adnotam/solidus_print_invoice',
    branch: 'custom'

# Autoprefixer
gem 'autoprefixer-rails'

# Materal Design Lite (CSS / JS)
gem 'material_design_lite-sass'

# For easy Horoscope-handling
gem 'zodiac'

# I18n
gem 'i18n-globals'
gem 'i18n-js', '>= 3.0.0.rc8'
gem 'rails-i18n', '~> 5.1'

# Use Rack CORS for handling Cross-Origin Resource Sharing (CORS), making cross-origin AJAX possible
gem 'rack-cors'

# RSS Parser
gem 'feedjira'
gem 'loofah'

# Countries info
gem 'countries'
gem 'country_select'
gem 'i18n_data'
gem 'tzwhere'

# User Authentication
gem 'devise'
gem 'devise-i18n'
gem 'doorkeeper'
gem 'doorkeeper-i18n',
    github: 'doorkeeper-gem/doorkeeper-i18n',
    ref: '43973fd2deea62817d48403f680f7e7ef5660f82'
gem 'doorkeeper-jwt'
gem 'omniauth-oauth2', '~> 1.3.1'
gem 'simple_token_authentication', '~> 1.14.0'

# Calendar
gem 'google-api-client', '~> 0.13.5', require: 'google/apis/calendar_v3'
gem 'omniauth-google-oauth2'

# Deezer
gem 'omniauth-deezer',
    github: 'adnotam/omniauth-deezer',
    branch: 'custom-deezer'

# Role and permission management
gem 'cancancan'
gem 'rolify'
gem 'solidus_user_roles',
    github: 'adnotam/solidus_user_roles'

gem 'enum_help'
gem 'simple_form'

# Enum support
gem 'active_record_enumerated_type'

# Display inline SVGs (Horoscope)
gem 'inline_svg'

# Tracking
gem 'ahoy_matey', '~> 2'

gem 'activeresource'

# Charts
gem 'chartkick'
gem 'groupdate'

# mail templates
gem 'foundation_emails'
gem 'inky-rb', require: 'inky'
gem 'nokogiri' # used by premailer-rails
gem 'premailer-rails'

gem 'wice_grid',
    github: 'patricklindsay/wice_grid',
    tag: 'v4.0.1'

# Amazon Search
gem 'amazon-ecs'
gem 'kaminari'
gem 'kaminari-i18n'

gem 'monetize',
    github: 'adnotam/monetize',
    branch: 'update_money_version'

gem 'money',
    github: 'adnotam/money',
    ref: 'a59f2bd0eb151187225f1531a0e26e09d26df0d9'

gem 'omniauth-amazon'

# to run background jobs
gem 'sidekiq'

# Find ip-based location
gem 'geocoder'
gem 'maxminddb'

gem 'streamio-ffmpeg'

gem 'mailkick'

gem 'originator'

# Parse spreadsheets
gem 'roo', '~> 2.7.1'

# Spam protection
gem 'invisible_captcha'

group :development, :test do
  # Call 'byebug' anywhere in the code to stop execution and get a debugger console
  gem 'byebug', platforms: %i[mri mingw x64_mingw]

  gem 'brakeman', require: false
  gem 'bullet'
  gem 'factory_bot_rails'
  gem 'faker'
  gem 'i18n-tasks'
  gem 'overcommit'
  gem 'rails-controller-testing'
  gem 'rspec-rails'
  gem 'rubocop', require: false

  gem 'rails_real_favicon'

  # Use Capistrano for deployment
  gem 'capistrano-rails'
  gem 'capistrano-rbenv'
  gem 'capistrano-yarn'
  gem 'capistrano3-puma'

  gem 'webmock'
end

group :development do
  # Access an IRB console on exception pages or by using <%= console %> anywhere in the code.
  gem 'listen', '>= 3.0.5', '< 3.2'
  gem 'web-console', '>= 3.3.0'
  # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
  gem 'spring'
  gem 'spring-commands-rspec'
  gem 'spring-watcher-listen', '~> 2.0.0'

  gem 'letter_opener'
end

group :test do
  gem 'capybara'
  gem 'capybara-screenshot'
  gem 'simplecov', require: false
end

# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: %i[mingw mswin x64_mingw jruby]
mattheworiordan commented 6 years ago

@afdev82 well thanks for the Gem file, but quite honestly if I cannot reproduce the problem easily, I just don't have the time to not only recreate the problem, but also fix it. If anyone else comes across this issue and wants some help, please do help me to reproduce and I'll help you get it fixed!

afdev82 commented 6 years ago

ok, we will see, thank you for the answer ;)

onomojo commented 6 days ago 
Capybara::Screenshot.after_save_screenshot do |path|
  WebMock.disable! # Allows the outgoing PUT to AWS
end