fiuba08 / robotframework

Automatically exported from code.google.com/p/robotframework
Apache License 2.0
0 stars 0 forks source link

`Pass Execution` in suite teardown causes tests to be reported failed on console #1656

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
Using Robot 2.8.4

Here is a very simple test:

*** Settings ***
suite teardown  pass execution  this is a test

*** test cases ***
mytest
    no operation

You expect this test to be OK, and here what we get on console:

[MBP]$ pybot test.txt
==============================================================================
Test
==============================================================================
mytest                                                                | PASS |
------------------------------------------------------------------------------
Test                                                                  | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================

According to console, it looks like the teardown failed somehow.
And the exit code is 1, indicating an error.
But when I open the report, the test is considered as OK. No failure at all. 
All green. Which is what I expect.

So there is a bug in the console output in that very specific case.

Original issue reported on code.google.com by laur...@bristiel.com on 12 Feb 2014 at 5:27

GoogleCodeExporter commented 9 years ago

Original comment by pekka.klarck on 13 Feb 2014 at 6:05

GoogleCodeExporter commented 9 years ago
This issue was updated by revision 874a1e732c8b.

The problem was that PassExecution exception was not handled correctly in suite 
teardowns. Status written to the XML output was correct, and thus logs/reports 
generated based on them were correct too. Statistics collected at runtime and 
shown on console were wrong. You would also see the problem if you executed 
tests programmatically. The received Result object could contain incorrect test 
statuses.

Pass Execution was already tested in suite teardown but this bug slipped 
through because the tests only verified statuses based on the XML output. Added 
test to verify statitics on the console too.

Original comment by pekka.klarck on 13 Feb 2014 at 6:12