Take screenshot automatically on failure (and also allow using other actions)

[GoogleCodeExporter]

I would be good to have flag/option that will automatically make screen shot on 
Fail or other unexpected behavior. Also it would be good to highlight object 
which is under testing e.x. for example object which is provided as location 
for validation/input.  

Original issue reported on code.google.com by Marcin.Koperski on 5 Oct 2010 at 9:12

[GoogleCodeExporter]

Hi Marcin,

If a test fails, you can log a screenshot:

Run Keyword If Test Failed  Capture Screenshot

(or Capture Page Screenshot), see [1] and [2].

Put that into the Test Teardown like:

*** Settings ***
Test Teardown  Teardown Selenium

*** Keywords ***
Teardown Selenium
  Run Keyword If Test Failed  Log Screenshot

Log Screenshot
  # to work around IE problems, we exclude the related browser profiles
  Run Keyword If  '${BROWSER_PROFILE}' != '*iexplore' and '${BROWSER_PROFILE}' != 'ie' and '${BROWSER_PROFILE}' != 'internetexplorer' and '${BROWSER_PROFILE}' != '*iehta' and '${BROWSER_PROFILE}' != 'iehta'  Capture Page Screenshot

So that would cover all cases where the test fails.

About highlighting the "object under test", I assume you mean something like 
draw a green border around a certain element in the page? Do you have exact 
ideas or an example of what you would like to do?

[1] 
http://robotframework-seleniumlibrary.googlecode.com/hg/doc/SeleniumLibrary.html
?r=2.4#Capture%20Screenshot

[2] 
http://robotframework-seleniumlibrary.googlecode.com/hg/doc/SeleniumLibrary.html
?r=2.4#Capture%20Page%20Screenshot

Original comment by spielman...@googlemail.com on 5 Oct 2010 at 2:57

[GoogleCodeExporter]

Hi Robert

My idea was to have similar look as we get using Selenium IDE, using button 
Find which highlights Target, maybe its possible to execute this command (or 
functionality ) and at the same time make screen shot.

Your solution is good, but it will give only screen shot on the finish of the 
test, sometimes you want to proceed even when validation failed but functional 
step are working properly.

Original comment by Marcin.Koperski on 5 Oct 2010 at 3:51

[GoogleCodeExporter]

This is a very often requested feature. Ought to be relatively easy to 
implement with some metaclass magic so that existing methods don't need to be 
touched.

Should we take screenshots always when a keyword fails automatically or should 
this be configurable? If this is configurable, should the default be on or off? 
Would it be enough to have an import parameter or should we have a keyword 
instead (or in addition)?

Original comment by pekka.klarck on 21 Oct 2010 at 10:50

• Changed state: Accepted
• Added labels: Priority-High, Target-2.4.1
• Removed labels: Priority-Medium

[GoogleCodeExporter]

I think it should be configurable (default on) using import parameter. The same 
solution would be nice to work with highlighting Target.

Original comment by Marcin.Koperski on 22 Oct 2010 at 6:41

[GoogleCodeExporter]

Highlighting target should be a separate issue. How to implement that depends 
on how Selenium RC supports highlighting.

Original comment by pekka.klarck on 22 Oct 2010 at 2:56

• Changed title: Take screenshot automatically when keyword fails

[GoogleCodeExporter]

I remember that we had major issues with screenshots in Internet Explorer, 
which effectively forced us to build a smart keyword that only takes a 
screenshot if no IE browser profile is being used for the test run. I'll 
investigate this, because even if taking the screenshot is configurable, it 
would probably not be desirable to switch screenshots on and off by ugly hacks 
if you run tests across multiple browsers.

Original comment by spielman...@googlemail.com on 24 Oct 2010 at 2:40

[GoogleCodeExporter]

Original comment by janne.t....@gmail.com on 12 Nov 2010 at 7:01

• Added labels: Target-2.5

[GoogleCodeExporter]

I've been investigating this and already got the basic functionality working. 
There's currently autoscreenshot.py file in the code base which implements a 
metaclass that automatically wraps keywords so that a screenshot is taken upon 
failure. This code isn't yet used, though.

Original comment by pekka.klarck on 14 Nov 2010 at 9:30

• Changed state: Started

[GoogleCodeExporter]

Since there are some differences in how taking screenshots works on different 
platforms, we decided to implement this so that it is possible to configure a 
keyword which is executed when another SeleniumLibrary keyword fails.

The keyword to be run can be given on library init or using a separate keyword-

Original comment by janne.t....@gmail.com on 15 Nov 2010 at 10:48

• Changed title: Allow execution of a keyword (such as Capture Screenshot) when keyword fails

[GoogleCodeExporter]

Implemented and added tests for the functionality described in comment 9 in 
revision d4feacc481f5.

The keyword that changes the keyword that is executed on failure is called `Run 
On Failure`.

Documentation is still missing.

Original comment by janne.t....@gmail.com on 15 Nov 2010 at 12:28

[GoogleCodeExporter]

Added some docs in revision d524080e96d5

Original comment by janne.t....@gmail.com on 15 Nov 2010 at 2:01

[GoogleCodeExporter]

Added some docs in revision d524080e96d5

Original comment by janne.t....@gmail.com on 15 Nov 2010 at 2:01

[GoogleCodeExporter]

I enhanced the docs in revision 3c2e1443ee. Janne, could you do the final 
review and close this one if everything looks good? Comments from others are 
obviously highly welcome too.

Original comment by pekka.klarck on 15 Nov 2010 at 11:18

• Changed state: Review

[GoogleCodeExporter]

Original comment by pekka.klarck on 15 Nov 2010 at 11:20

• Changed title: Take screenshot automatically on failure (and also allow using other actions)

[GoogleCodeExporter]

Added one more example in revision  ab1c024223ea

I consider this now done.

Original comment by janne.t....@gmail.com on 16 Nov 2010 at 5:56

• Changed state: Done

[GoogleCodeExporter]

I think this solution is useful, not only for selenium test, because often if 
something brakes there is need to make additional steps

Original comment by Marcin.Koperski on 16 Nov 2010 at 7:36

[GoogleCodeExporter]

Notice that this feature only works with Selenium. Generally you should use 
teardowns if you need to do actions in case of failures.

Original comment by pekka.klarck on 16 Nov 2010 at 8:27

[GoogleCodeExporter]

This issue was updated by revision 95cc7d834c.

I fixed a mini-typo in the docs. Looks good otherwise.

Original comment by spielman...@googlemail.com on 16 Nov 2010 at 9:15

[GoogleCodeExporter]

This issue was updated by revision d43b5fe19f.

I fixed another mini-typo in the docs. =)

Original comment by spielman...@googlemail.com on 16 Nov 2010 at 9:16

[GoogleCodeExporter]

anybody knows why this function throw out the error?
----
ERROR: java.io.FileNotFoundException: 
/var/folders/yC/yC-BR3EaE+0V23rq+scaok+++TI/-Tmp-/customProfileDir9b54401dac2c40
f19022e892b0290b79/screenshots/page-screenshot-9b54401dac2c40f19022e892b0290b79.
png (No such file or directory)
----
java version "1.6.0_20"
Java(TM) SE Runtime Environment (build 1.6.0_20-b02-279-10M3065)
Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01-279, mixed mode)
----
Selenium RC server v2.0 a7  and selenium library 2.5 ;Robot Framework 2.5.4 
(Python 2.6.1) 

Original comment by jollych...@gmail.com on 29 Nov 2010 at 7:59