Results are not recorded for Scenario Outline when using --tags @tag,~@otherTag

[kutzi]

I've tracked down an issue with Serenity Cucumber that it does support (to some point) the .@tag,~@otherTag syntax in that it will run the scenario outline, but then fail to record the results.

The issue is in SerenityReporter.examplesMatchFilter() that the TagExpressionParser doesn't seem to support ',~' as being the same as 'and not' and so always returning false for expressionNode.evaluate()

Consequently exampleCount will stay 0 and in finishExample() exampleCount will never be 0 in the if check - as it will already be -1 after the first call and then be decremented even further.

[kutzi]

As cucumber-jvm reports the '~' syntax to be deprecated anyway, I'll just fix my code to use the 'and not' syntax and not trying to generate a minimal test project to recreate the issue

[kutzi]

But maybe you still want to have a look to fix the issue and/or warn users (even more) that the '~' syntax must be avoided.

[wakaleo]

I believe the tag processing uses the Cucumber parser (i.e. Serenity does not do any parsing itself), so it may be a Cucumber bug.

[kutzi]

Agreed. But it also only seems to surface in the combination of Serenity-Cucumber and Cucumber. Because I could see that the tests are executed and recorded in the Cucumber part (e.g. when enabling the junit formatter in cucumber), but missing from the SERENITY-JUNIT reports and ergo in the BDD reports

[kutzi]

FYI: the TagExpressionsParser seems to be specifically targetted at the new style expressions, only. See https://github.com/cucumber/tag-expressions-java and https://docs.cucumber.io/tag-expressions/

[cliviu]

Hi @kutzi , thanks for spotting this. Until we have time to take a look at the issue, please switch to the new tag syntax if is possible. Thanks !

[wakaleo]

Indeed, Serenity-cucumber currently only supports the new tag syntax: see https://github.com/cucumber/cucumber/tree/master/tag-expressions.