mattheworiordan / capybara-screenshot

Automatically save screen shots when a Capybara scenario fails
MIT License
1.02k stars 168 forks source link

Not getting any image on error although path specfied #66

Open justin808 opened 10 years ago

justin808 commented 10 years ago

On rspec errors, I get a message

Screenshot: [some file path]

However, there's no file.

I've got the following gem versions:

capybara (2.1.0)
  mime-types (>= 1.16)
  nokogiri (>= 1.3.3)
  rack (>= 1.0.0)
  rack-test (>= 0.5.4)
  xpath (~> 2.0)
capybara-screenshot (0.3.14)
  capybara (>= 1.0, < 3)
  launchy
sethcall commented 10 years ago

I'm also seeing the same thing. Ruby 2.0 and poltergeist 1.5.0

capybara (2.2.0)
    capybara-screenshot (0.3.14)
      capybara (>= 1.0, < 3)
    capybara-webkit (1.1.0)
      capybara (~> 2.0, >= 2.0.2)
      capybara (>= 1.1.2)
      capybara (~> 2.1)
poltergeist (1.5.0)
sethcall commented 10 years ago

After debugging it a little, i see that the save method in capybara-screenshot has this:

return if capybara.current_path.to_s.empty?

This, in my case, is empty. So, save bails, but later the code that prints the path still executes, resulting in the confusing situation. I'm going to double-check soon whether that the path really is or isn't valid. (is it my test, or capybara?)

sethcall commented 10 years ago

Ok, figured it out. I was doing:

config.after(:each) do
  Capybara.reset_sessions!
end 

Instead if I do this, reset_sessions! goes after the screenshot gem. Problem fixed.

config.append_after(:each) do 
  Capybara.reset_sessions!
end
mattheworiordan commented 10 years ago

Ah, good spot @sethcall, that would make sense. Do you think I can close this as an issue then as this is not really something we can prevent i.e. if someone adds an after block that runs before capybara-screenshot

RSO commented 10 years ago

I have the same error locally, and the only config.after I have is this:

config.after(:each) do
  DatabaseCleaner.clean
end

My Gemfile.lock:

    poltergeist (1.5.0)
      capybara (~> 2.1)
      cliver (~> 0.3.1)
      multi_json (~> 1.0)
      websocket-driver (>= 0.2.0)
    capybara (2.2.1)
      mime-types (>= 1.16)
      nokogiri (>= 1.3.3)
      rack (>= 1.0.0)
      rack-test (>= 0.5.4)
      xpath (~> 2.0)
    capybara-screenshot (0.3.16)
      capybara (>= 1.0, < 3)
      launchy

Will try to do some debugging

sethcall commented 10 years ago

Welll.... It seems to me if the save operation fails, ideally it would say why... And additionally it would be nice if the msg about where the file was saved too was not emitted on an error case, since it wasn't actually saved. (this was a draft I never hit send on, and Remon's email just made me see it)

Seth On Dec 28, 2013 2:40 PM, "mattheworiordan" notifications@github.com wrote:

Ah, good spot @sethcall https://github.com/sethcall, that would make sense. Do you think I can close this as an issue then as this is not really something we can prevent i.e. if someone adds an after block that runs before capybara-screenshot

Reply to this email directly or view it on GitHubhttps://github.com/mattheworiordan/capybara-screenshot/issues/66#issuecomment-31304403 .

RSO commented 10 years ago

Ok, so I found the source of all evil: Capybara itself. In lib/capybara/cucumber.rb you can find the following lines:

After do
  Capybara.reset_sessions!
end

Any ideas on how I can make sure capybara-screenshot gets ran before that after block?

The cucumber wiki says the following about After hooks:

After hooks will be run after the last step of each scenario, even when there are failing, undefined, pending or skipped steps. They will run in the opposite order of which they are registered.

Which leads me to think I should require capybara-screenshot after capybara itself, which I think is the case already.

RSO commented 10 years ago

Ok, forget everything I said, the problems I received were because of a problem on my end. The acceptance test was operating out of an email (through capybara-email), which caused the screenshot_and_open_image command to fail silently.

I'm still in favor of removing the strange return in the Saver#save method, since it's very weird that that thing fails silently instead of just throwing an error. I'll try to work out a pull request later today!

AlexVPopov commented 10 years ago

I also have this issue - I only get an html file, but no png. As I see this could be caused by a variety of reasons. Would you give me any tips how to investigate this on my end?

RSO commented 10 years ago

I debugged this by adding a lot of puts statements in my gems (open them with bundle open GEMNAME). You should probably begin with debugging the capybara_screenshot#save method, through this method I found out that the page that should be opened was nil. etc. etc.

One of the main reasons that the page will be nil is either:

Hope this helps!

marcusg commented 10 years ago

Anyone know a workaround for this?

AlexVPopov commented 10 years ago

@marcusg I got this error when I used screenshot_and_open_image. If this is the same for you, just use screenshot_and_save_page. This way I always get a .png as well. Alternatively, if you are on a Mac, also make sure you have Preview (or whatever your default .png viewer is) opened during execution.

RSO commented 10 years ago

@marcusg did you try the things I mentioned in the comment above yours? And if so, could you give some feedback on where you're stuck?

marcusg commented 10 years ago

@AlexVPopov In my case both screenshot_and_open_image and screenshot_and_save_page won't create a *.png file (I'm using Linux Mint).

@RSO I'll try to add some debugging puts in the next days.... I'm using capybara-email too, but not within the failing specs, where I need the screenshot ability.

Thanks!

mattheworiordan commented 10 years ago

Hi all, we should definitely add some logging if the screenshots cannot be saved. I will look at making a change to support that.

oesgalha commented 10 years ago

I had the same problem and did something similar to @sethcall to solve it.

I was doing this in one of my tests:

after(:each) { page.driver.reset! }

(which is a workaround for a common issue to those who use phantomjs/poltergeist/capybara: jonleighton/poltergeist#375) And after switching it to:

append_after(:each) { page.driver.reset! }

The screenshots are back :thumbsup:

Thanks @sethcall !

axlekb commented 10 years ago

I see the same issue with capybara 2.4.1 combined with poltergeist 1.5.1. I see the following error when using screenshot_and_open_image:

Launchy::ApplicationNotFoundError:
       No application found to handle '/path/to/image/location.png'

However, that seems to be completely correct, because that file does not exist. However there is a .html.

Selenium driver works. Webkit driver works. Just not poltergeist.

Anyone else figure this out?

axlekb commented 10 years ago

After a little more research it seems that using poltergeist's page.save_screenshot works when :full => trueis not specified: page.save_screenshot('/path/to/image.png') however does not generate a file with the full option: page.save_screenshot('/path/to/image.png', :full => true))

My workaround has been to put the snipped below into my capybara configuration file which overrides capybara-screenshot.rb#L94

Capybara::Screenshot.class_eval do
  register_driver(:poltergeist) do |driver, path|
      driver.render(path)
   end
end
axlekb commented 10 years ago

I should also note that I'm also not receiving a file location on failure. With any driver. On 0.3.21.

mattheworiordan commented 10 years ago

@axlekb can you confirm which versions of Gems you are running, and ideally if you have any code that could replicate this issue it would help me immensely in implementing a fix.

axlekb commented 10 years ago
capybara (2.4.1)
capybara-screenshot (0.3.21)
poltergeist (1.5.1)
rspec-rails (3.0.2)

If no one else is reporting the issue, I'm not sure it's a big enough issue for me to spend time putting together some sample code.

justin808 commented 10 years ago

I'm definitely seeing the issue. I work around it by manually writing screenshots which is not ideal.

mattheworiordan commented 10 years ago

Would someone mind confirming if this issue still exists with version 1.0.1 of Capybara Screenshot?

axlekb commented 10 years ago

I can confirm that capybara-screenshot outputs the filenames in 1.0.1. :+1:

I still have the issue with poltergeist not outputting a .png with the default settings, however my workaround still works:

Capybara::Screenshot.class_eval do
  register_driver(:poltergeist) do |driver, path|
    driver.render(path)
  end
end
aguynamedben commented 10 years ago

Hi, thanks for this fantastic gem. I've been trying for about 2 hours to get automatic screenshots upon test failure working. screenshot_and_save_page and screenshot_and_open_image are working fine, but I can't get screenshots to work on tests that fail.

I put in the puts statements @RSO recommends above, and indeed capybara.current_path is blank, so Capybara::Screenshot::Saver#save is returning at the line return if capybara.current_path.to_s.empty?. I'm a little confused where to go from there though. I have a pretty normal looking Capybara test, so why would that value be blank? I don't understand why current_path would be empty, and I don't understand the comment in the code where the problematic line is.

I have searched for all the nasty reset things above and do not think I'm doing anything wrong there. Here is my code... thanks for any help you have to offer.

I'm using all webkit...

  * capybara (2.4.3)
  * capybara-screenshot (1.0.1)
  * capybara-webkit (1.3.0)
  * rspec (2.14.1)
  * rspec-core (2.14.7)
  * rails (3.2.17)

spec/features/features_helper.rb

require 'spec_helper'
require 'capybara/rspec'
require 'capybara-screenshot'
require 'capybara-screenshot/rspec'

Capybara.default_driver = :webkit
Capybara.javascript_driver = :webkit

Capybara.server_host = ENV['BASE']
Capybara.server_port = ENV['BASE_PORT']
Capybara.default_wait_time = 30

Dir[Rails.root.join("spec/features/support/**/*.rb")].each { |f| require f }

RSpec.configure do |config|
  config.include AcceptanceHelper, type: :feature
end

Capybara::Screenshot.webkit_options = { width: 1280, height: 1024 }
Capybara.save_and_open_page_path = ENV['CIRCLE_ARTIFACTS'] if ENV['CIRCLE_ARTIFACTS'].present?

The spec

    context "if the user has an account" do
      let!(:user) { create(:user, :authorized, provider: 'facebook', uid: '12345') }

      it "can login" do
        visit root_path
        click_link "Login using Facebook"

        page.should have_content "Ask GOOD Question"
        page.should have_css("#feed")
        expect(current_path).to eq root_path
      end
    end

The spec failing as expected

  1) Login when user authorizes the app if the user has an account can login
  Failure/Error: page.should have_content "Ask GOOD Question"
    expected to find text "Ask GOOD Question" in "Ask Question Add Post Add Link Sorry, but there were no results."
  # ./spec/features/login_spec.rb:24:in `block (4 levels) in <top (required)>'
  # ./spec/support/subdomains_spec_helper.rb:45:in `block (3 levels) in <module:SubdomainsSpecHelper>'
  # ./app/models/tenant.rb:36:in `with_tenant'
  # ./spec/support/subdomains_spec_helper.rb:43:in `block (2 levels) in <module:SubdomainsSpecHelper>'
  # ./app/models/tenant.rb:36:in `with_tenant'
mattheworiordan commented 10 years ago

@aguynamedben I will take a look and see if I can replicate this, however it would be really useful if you could replicate it in a simple Rails / Ruby app so that I can see the issue. Let me see what I can find, generally people have complained about Capybara.current_path being empty when they have a reset of some sort. Can I ask you also, if you put a debugger in the one of your features and check the value of Capybara.current_path is it nil then, or is it only nil once capybara-screenshot kicks in?

mattheworiordan commented 10 years ago

@axlekb out of interest, if you change your workaround code do the PNG screen shots stop working again?

Capybara::Screenshot.class_eval do
  register_driver(:poltergeist) do |driver, path|
    driver.render(path)
  end
end
axlekb commented 10 years ago

@mattheworiordan, not exactly sure what you're asking, but yes, if I do not have the code you have (also below), then the output logs that the PNG was created, but no PNG exists.

Capybara::Screenshot.class_eval do
  register_driver(:poltergeist) do |driver, path|
    driver.render(path)
  end
end
mattheworiordan commented 10 years ago

@axlekb, to be honest I have no idea why the full argument is there in the first. It was introduced back in Nov 2011 and I was personally not using Poltergeist at the time, so I accepted the PR. I am tempted to just rip it out and see if people start reporting issues. Or perhaps, to be safer, I could sync it on with newer version of Poltergeist, what version are you on? See the commit that introduced that param here

karlingen commented 9 years ago

@axlekb @mattheworiordan I can confirm that axlekb's workaround fixed it for me as well. Let me know if I can help you guys any further into investigating what the issue might be.

Running on this stack:

capybara (2.4.4)
rspec (3.1.0)
poltergeist (1.5.1)
capybara-screenshot (1.0.3)
phantomjs (1.9.7.1)
mattheworiordan commented 9 years ago

@axlekb would you mind trying the Gem using branch poltergeist-fix without the workaround to see if it works for you? You should be able to do something like this in your Gemfile: gem 'capybara-screenshot', git: 'https://github.com/mattheworiordan/capybara-screenshot', branch: 'poltergeist-fix'

axlekb commented 9 years ago

@mattheworiordan - The FULL page screenshot works! On failure, the png/html file locations are not printed to the output though. [Edit: make sure you see post below]

axlekb commented 9 years ago

Actually - Hold that. I'm actually getting full page PNG screenshot with 1.0.3 and 1.0.4 without the workaround. Still missing png/html file locations.

With branch: 'poltergiest-fix', getting a screenshot, but it's cut-off and no file location.

capybara (2.4.4)
rspec (3.1.0)
poltergeist (1.5.1)
capybara-screenshot (1.0.3 or 1.0.4)
phantomjs (1.9.7)

@karlingen - what issue were you having?

For me, the only remaining issue is the file locations in the output.

karlingen commented 9 years ago

@axlekb the issue I had before your workaround was that it did not save any png file, eventhough I called screenshot_and_save_page. What do you mean with "missing png/html file locations."? Are they supposed to be written out when running the specs?

axlekb commented 9 years ago

@karlingen On test failures in rspec, screenshot_and_save_page is automatically called and previously the filename was output as well.

karlingen commented 9 years ago

@axlekb Oh, I have not noticed that.

mattheworiordan commented 9 years ago

@axlekb can you grab me a sample of your test output without the screen shots i.e. where it fails please? I will try and replicate soon.

axlekb commented 9 years ago

Here's the output. Both screenshots were generated:

$ rspec spec/features/unauthorized_user_sees_error_spec.rb 

Failures:

  1) Dummy Test to see .png and .html output
     Failure/Error: expect(1).to eq(0)

       expected: 0
            got: 1

       (compared using ==)
     # ./spec/features/unauthorized_user_sees_error_spec.rb:11:in `block (2 levels) in <top (required)>'

Finished in 34.06 seconds (files took 27.28 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/features/unauthorized_user_sees_error_spec.rb:8 # Dummy Test to see .png and .html output

Top 1 slowest examples (31.99 seconds, 93.9% of total time):
  Dummy Test to see .png and .html output
    31.99 seconds ./spec/features/unauthorized_user_sees_error_spec.rb:8

Randomized with seed 52215