Spock gets confused if "when" test cases look similar.

GoogleCodeExporter commented 8 years ago

Let's take this test-case:
----------------------------------------------------
import static org.junit.Assert.assertEquals
import spock.lang.*

@Unroll 
class ShorteningTextTest extends Specification {

    @IgnoreRest
    def "Short sentence #text shortening by #amount "(){
        expect:
            text==result
        where:
            text                                    | amount|| result
            "<p>Elip</p>s.<p></p>p"                 | -1    || "<p>Elip</p>s.<p></p>p"
            "<p>Elip"                               | -1    || "<p>Elip"
            "<p>Elip."                              | -1    || "<p>Elip."
            "<p>Elip."                              | -2    || "<p>Elip."
            "<p>Eli.."                              | -1    || "<p>Eli.."
            "<p>Eli.. . "                           | -1    || "<p>Eli.. .</p>"
            "<p>Eli.    A"                          | -1    || "<p>Eli.</p>"
            "<p>Eli.    A."                         | -1    || "<p>Eli.</p>"
            "<p>Eli.    A."                         | -2    || "<p>Eli. A."
            "<p>Eli.<br> A."                        | -2    || "<p>Eli.<br> A."
            "<p>Eli.<br> A. "                       | -2    || "<p>Eli.<br> A.</br></p>"
            "<p>Eli.</p>p"                          | -1    || "<p>Eli.</p>p"
            "<p>Eli.</p>P"                          | -1    || "<p>Eli.</p>"
            "<p>Eli.</p>p"                          | -2    || "<p>Eli.</p>p"
            "<p>Eli.</p>P"                          | -2    || "<p>Eli.</p>P"
            "<p>Eli</p>p.<p>"                       | -1    || "<p>Eli</p>p."
            "<p>Elip.</p><p>"                       | -1    || "<p>Elip.</p>"
            "<p>Elip</p>.<p>"                       | -1    || "<p>Elip</p>."
            "<p>Elip</p>.<p>x"                      | -1    || "<p>Elip</p>.<p>x"
            "<p>Elip.</p><p>."                      | -1    || "<p>Elip.</p><p>."
            "<p>Elip</p>.<p>P"                      | -1    || "<p>Elip</p>."
            "<p>Elip.</p><p>P."                     | -1    || "<p>Elip.</p>"
            "<p>Elip</p><p>."                       | -1    || "<p>Elip</p><p>."
            "<p>Elip.</p><p>."                      | -1    || "<p>Elip.</p><p>."
            "<p>Elip.</p><p>"                       | -1    || "<p>Elip.</p>"
            "<p>Elip</p><p>. "                      | -1    || "<p>Elip</p><p>.</p>"
            "<p>Elip.</p><p>. "                     | -1    || "<p>Elip.</p><p>.</p>"
            "<p>Elip.</p><p>."                      | -1    || "<p>Elip.</p><p>."
            "<p>Elip.</p><p>. P"                    | -1    || "<p>Elip.</p><p>.</p>"
            "<p>Elip</p><p>. P"                     | -1    || "<p>Elip</p><p>.</p>"
            "<h1><p>Elip.</p></h1>Follow"           | -1    || "<h1><p>Elip.</p></h1>"
            "<h1><p>Elip</p>.</h1>Follow"           | -1    || "<h1><p>Elip</p>.</h1>"
            "<h1><p>Elip.</p>.</h1>Follow"          | -1    || "<h1><p>Elip.</p>.</h1>"
            "<h1><p>Elip.</p></h1>.Follow"          | -1    || "<h1><p>Elip.</p></h1>."
            "<h1><p>Elip.</p></h1>.Follow"          | -2    || "<h1><p>Elip.</p></h1>.Follow"
            "<h1><p>Elip.</p></h1>"                 | -1    || "<h1><p>Elip.</p></h1>"
            "<h1><p>Elip</p>.</h1>"                 | -1    || "<h1><p>Elip</p>.</h1>"
            "<h1><p>Elip.</p>.</h1>"                | -1    || "<h1><p>Elip.</p>.</h1>"
            "<head><h1><p>Elip.</p>.</h1>.</head> " | -1    || "<head><h1><p>Elip.</p>.</h1>.</head>"
            "<h1><p>Elip.</p>P.</h1>"               | -1    || "<h1><p>Elip.</p></h1>"
            "<h1><p>Elip.</p></h1>P"                | -1    || "<h1><p>Elip.</p></h1>"
            "<h1><p>Elip.</p></h1>.P"               | -1    || "<h1><p>Elip.</p></h1>."
            "<h1><p>Elip.</p></h1>."                | -1    || "<h1><p>Elip.</p></h1>."
            "<h1><p>Elip.</p></h1>P."               | -1    || "<h1><p>Elip.</p></h1>"
            "<h1><p>Elip.</p></h1>P."               | -2    || "<h1><p>Elip.</p></h1>P."
            "<i><h1><p>Elip.</p></h1>P.</i> "       | -2    || "<i><h1><p>Elip.</p></h1>P.</i>"
            "<p>El.i.p</p>"                         | -1    || "<p>El.i.p</p>"
            "<p>El.i.</p>p"                         | -1    || "<p>El.i.</p>p"
            "<p>El.i.</p>P"                         | -1    || "<p>El.i.</p>"
            "<p>Elip.</p>S<p></p>"                  | -1    || "<p>Elip.</p>"
            "<p>Elip</p>s.<p></p>"                  | -1    || "<p>Elip</p>s."
            "<p>Elip</p>s.<p></p>"                  | -1    || "<p>Elip</p>s."
            "<p>Elip</p>s<p>.</p>"                  | -1    || "<p>Elip</p>s<p>.</p>"
            "<p>Elip</p>s<p>.</p>P"                 | -1    || "<p>Elip</p>s<p>.</p>"
            "<p>Elip</p>s<p>.</p>p"                 | -1    || "<p>Elip</p>s<p>.</p>p"
            "<p>Elip</p>s.<p></p>P"                 | -1    || "<p>Elip</p>s."
            "<p>Elip</p>s<p>.</p>P"                 | -1    || "<p>Elip</p>s.<p></p>"
            """al<p class="ala" >El.</p>"""         | -1    || """al<p class="ala" >El.</p>"""
            """al<p class="ala" >El.I</p>"""        | -1    || """al<p class="ala" >El.</p>"""
            """al<p class="ala" >E.</p>  """        | -1    || """al<p class="ala" >E.</p>"""
            """<ul><li>ela<br>p.I dalej</li></ul>"""| -1    || """<ul><li>ela<br>p.</br></li></ul>"""
            """<ul>ela<br>p</ul>.A slslsl"""        | -1    || """<ul>ela<br>p</ul>."""
            """<ul>ela<br>p</ul>a.<p>Slslsl"""      | -1    || """<ul>ela<br>p</ul>a."""
            """<ul>ela<br>p.</ul>A slslsl"""        | -1    || """<ul>ela<br>p.</br></ul>"""
    }
}
----------------------------------------------------------
Now look at the attached image. The argument value reported by eclipse does not 
match one defined in the file. Looks like the values of arguments and results 
are mixed in some way.

What version of Spock and Groovy are you using?
Spock 0.7 Groovy 1.8.3 Eclipse Indigo SP2

Please provide any additional information below. You can also assign
labels.

Original issue reported on code.google.com by dsmo...@gmail.com on 15 Apr 2013 at 2:40

Attachments:

image.png

GoogleCodeExporter commented 8 years ago

I don't think this report is enough to consider this as a bug. First of all, 
the screenshot only shows how Eclipse presents test results. Your test class is 
extremely hard to read (it's definitely not minimal reproduction steps) and you 
did not specify what you think should be the correct result.

For me (Idea 13, 64-bit Java 1.7.0_45 on Linux) the GUI test report shows the 
line underlined on the screenshot in correct order.

Note that JUnit by default does not guarantee any order of execution of test 
methods: https://github.com/junit-team/junit/wiki/Test-execution-order

I would suggest rejecting this issues as "not a bug" because too much is 
unclear here.

Original comment by piotr.ku...@gmail.com on 1 Jan 2014 at 12:35

GoogleCodeExporter commented 8 years ago

Indeed surefire report for the same testcase results in correct order. This is 
probably eclipse specific and doesn't belong here.

Original comment by dsmo...@gmail.com on 1 Jan 2014 at 8:11

GoogleCodeExporter commented 8 years ago

Please close the issue.

Original comment by dsmo...@gmail.com on 1 Jan 2014 at 8:12

GoogleCodeExporter commented 8 years ago

Original comment by pnied...@gmail.com on 2 Jan 2014 at 2:14

Changed state: WontFix

djcrankypants / spock

Spock gets confused if "when" test cases look similar. #309