Scenario Outlines inconsistencies

Behat / Gherkin

Gherkin parser, written in PHP for Behat project

MIT License

1.04k stars 88 forks source link

Scenario Outlines inconsistencies #153

Open ciaranmcnulty opened 5 years ago

ciaranmcnulty commented 5 years ago

Checking the parsing examples from Gherkin, the following fail due to issues with scenario outlines:

~descriptions.feature - the description on the final Example causes a parse error~ [moved to #154]
incomplete_scenario_outline (and readme_example, scenario_outlines_with_tags, several_examples, tags) - scenario outlines without an Example are not allowed by our parser but are just ignored by cucumber
- scenario_outline - Looks like Scenario keyword can be used for outlines now?

ciaranmcnulty commented 5 years ago

Checked with @aslakhellesoy and in Gherkin 6 Scenario Outline and Scenario are synonyms - they only differ based on whether they are followed by an Example, so the following are valid:

Scenario Outline: Something
   Given I don't have any examples

Scenario: Something else
    Given <foo>

Examples:
    | foo |
    | bar |

stof commented 5 years ago

This would require a BC break here, and in Behat though, as outlines are currently treated differently from scenarios in Behat.

Btw, I'm wondering how cucumber deals with the output formatting for these cases. Does it still tries to output something looking like a colored gherkin file ?

ciaranmcnulty commented 5 years ago

I don't think it's necessarily a BC break - it allows some new syntaxes (above) that previously would have been a parse error

(our AST would need to still differentiate ScenarioOutline nodes from Scenario nodes, but we could detect which it is based on whether it is followed by Examples)

stof commented 5 years ago

ah indeed, if we keep our 2 different nodes based on the presence of Examples, there is no BC break.

ciaranmcnulty commented 3 years ago

@dpakach As you've been looking at how Examples and Outlines are treated, what do you think about implementing this?

aslakhellesoy commented 3 years ago

Just my 2c - the official Gherkin parser does the same - the AST node that gets created depends on the absence/presence of examples. And we managed to release this without any users complaining about BC issues.

ciaranmcnulty commented 3 years ago

Removing these lines will now produce failing tests

https://github.com/Behat/Gherkin/blob/master/tests/Behat/Gherkin/Cucumber/CompatibilityTest.php#L33-L35