Scenario outline does not behave in accordance with documentation

[hicksjduk]

The documentation of pytest-bdd states that "Gherkin scenario outlines are supported by pytest-bdd exactly as it's described in be behave docs". However, there is one situation in which the support for scenario outlines is different from what behave supports.

Sometimes, when defining a scenario outline, it is useful to include additional columns in the "Examples" table, which are not referenced in any step definition - for documentation purposes, or to specify values that are not needed now, but will be used later. If you do this in behave, it correctly ignores such columns. However, if you do it in pytest-bdd an error is thrown.

For example, I have a scenario outline where the examples table contains the columns servicestring, errorMsg, logMsg and note - note being a column that is purely there for documentation purposes. The first three columns are referenced by step definitions, but note is not and should not be. When I run this test, I get the error:

Scenario "..." in the feature "..." has not valid examples. Set of step parameters ['errorMsg', 'logMsg', 'servicestring'] should match set of example values ['errorMsg', 'logMsg', 'note', 'servicestring'].

It might be that there is a way to overcome this through configuration, but if so it isn't clearly documented so it effectively isn't available. (And from looking at the code, it doesn't appear that there is.) If there isn't a way to overcome it, it is a serious error.

[hicksjduk]

This message is issued by the following method in parser.py:

    def validate(self):
        """Validate the scenario.

        :raises ScenarioValidationError: when scenario is not valid
        """
        params = self.params
        example_params = self.get_example_params()
        if params and example_params and params != example_params:
            raise exceptions.ScenarioExamplesNotValidError(
                """Scenario "{0}" in the feature "{1}" has not valid examples. """
                """Set of step parameters {2} should match set of example values {3}.""".format(
                    self.name, self.feature.filename, sorted(params), sorted(example_params)
                )
            )

I think that the problem is that params and example_params are expected to be the same. I can understand that we want to make sure that all the values in params are in example_params, but it is not reasonable to expect that all the values in example_params are in params - params should be allowed to be a subset of example_params.

So a way of fixing the error would be to replace

        if params and example_params and params != example_params:
            raise exceptions.ScenarioExamplesNotValidError(
                """Scenario "{0}" in the feature "{1}" has not valid examples. """
                """Set of step parameters {2} should match set of example values {3}.""".format(
                    self.name, self.feature.filename, sorted(params), sorted(example_params)
                )
            )

with

        if params and example_params and any(p not in example_params for p in params):
            raise exceptions.ScenarioExamplesNotValidError(
                """Scenario "{0}" in the feature "{1}" does not have valid examples. """
                """Set of step parameters {2} should be a subset of example values {3}.""".format(
                    self.name, self.feature.filename, sorted(params), sorted(example_params)
                )
            )