[feature request] use tags for example tables for scenarios with more example tables

[jimmy882]

When I have a scenario with more example tables with different tags:

Scenario Outline: Test example
  Given Step with <param1>
  When Step with <param2>
  Then Result is <result>

@smoke
Examples:
| param1 | param2 | result |
| foo    | bar    | foobar |
| 1      | 2      | 3      |

@regression
Examples:
| param1 | param2 | result |
| abc    | def    | abcdef |
| z      | y      | z      |

I would like to run only those examples that are tagged with specific tag, for example @smoke. This is possible with pure cucumber-js, do they use different tag-expressions? According to cucumber documentation: Tags can be placed above the following Gherkin elements:

• Feature
• Scenario
• Scenario Outline
• Examples

I tried to run cucumber-js and it worked, but when running cypress with cypress-cucumber-preprocessor, I get an error: We could not detect any tests in the above file. Write some tests and re-run.

[lgandecki]

Hey @jimmy882 . I believe we use the same tag-expressions, but since we are running things through cypress as basically dynamically-created mocha tests our implementation of how we deal with tags is custom (and a reason for lack of features and problems). I don't have any issues with what you are suggesting, and might get around to it one day, but it's not high on my list of priorities, since none of our clients requested this so far.

I'd love to take a PR though!

[Mark-J-Lawrence]

Hi @lgandecki , I'm wanting to run exactly the same scenario as @jimmy882 to express different outcomes in different environments, but have found the same problem.

I tried to trick it by setting a @all tag on the Scenario Outline and @e2e on the examples, and then set my TAGS env to @all and @e2e, but that didn't work either.

At the moment I'll have to copy the scenario outline and give it a different tag as a workaround

[satyasrikanth]

Looking for the same feature, is there any workaround, please?

[askojoke]

Also need this functionality with tagged examples. any news?

[AntonioGargaro]

Also looking for this functionality, bump.

[asanchezgiraldo]

We are also looking for this solution. Is there a beta version we can use to help test?

[phpworker]

we need that in our company!

[siparsons]

This feature would be great. bump

[giselerosa]

This feature would be very interesting, does anyone have any workarounds?

[kakamorsch]

It would be awesome to have this feature, did anyone thought on a workaround?

[devsrihari4]

this is what exactly I needed for a scenario. Appreciate help on this feature

[frnc07]

Any update on this feature request. It would be very helpful if this feature is implemented

[DesaiHimani]

This will be really helpful. Kindly share if there is any workaround for now.

[dkusakina]

If at some point this feature will be implemented, that would be really great, thank you!!

[091sameer]

We would really appreciate if this feature can be implemented. Kindly share if there is any workaround for now.

[RitikaGhosh15]

Hello.. Any update when will this be merged? Thank you for adding this feature in first place

[lgandecki]

Hello @RitikaGhosh15 , this is an issue not a PR, do you mean any specific PR you would like to be merged? The #557 ?

[RitikaGhosh15]

@lgandecki ohk, i assumed it to be a new feature. Actually i am trying to run tags attached to examples in cypress-cucumber-preprocessor with command but its not considering the example tag and running both the examples

npx cypress open --env env=devint TAGS='@mobile and @reg'

[Sonaryr]

This is exactly the usecase we had for our porject, that's why I made #557 to solve this, and in the mean while live up to the gherkin specs that says it should be possible.

[manojbharadwajZava]

Would be great to have this feature, currently there is no workaround :( If someone has one, please share!

[rkrisztian]

This is quite a huge setback for my project too. :(

[lgandecki]

there is work started here if you guys would like to contribute and help pushing this through the finish line: https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/pull/557

[rkrisztian]

@RitikaGhosh15, are you sure about the correctness of syntax of the command line you used?

npx cypress open --env env=devint TAGS='@mobile and @reg'

This will ignore your tags, thus should be:

npx cypress open --env env=devint --env TAGS='@mobile and @reg'

[RitikaGhosh15]

@RitikaGhosh15, are you sure about the correctness of syntax of the command line you used?

npx cypress open --env env=devint TAGS='@mobile and @reg'

This will ignore your tags, thus should be:

npx cypress open --env env=devint --env TAGS='@mobile and @reg'

Hi rkrisztian , I tried with the new command too but no success. with the new command its still considering @mobile and @smoke even when i pass TAGS='@mobile and @reg'

my requirement is if i pass TAGS='@mobile and @reg', it should pick example under reg and not smoke.

[lgandecki]

:tada: This issue has been resolved in version 4.2.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[RitikaGhosh15]

hello @lgandecki , As per the commit https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/commit/2789e0b434ec9a1a360e39d864d9e020f1b028ac if tags are reversed in position, will it still work? as for me if i remove tag from example 1 and put it on example 2 and run the command (npx cypress open --env env=dev --env TAGS='@desktop and @tag2'), its not working as expected.

Please help me to understand where am i going wrong

[jime2003]

I am also having the same issue as @RitikaGhosh15 Can someone please help?

[rkrisztian]

I did some tests with version 4.2.0, but I'm unable to reproduce the issue reported by @RitikaGhosh15. For me the tags were filtered correctly, no matter how I tried.

[jime2003]

I did some tests with version 4.2.0, but I'm unable to reproduce the issue reported by @RitikaGhosh15. For me the tags were filtered correctly, no matter how I tried.

@rkrisztian i am using below command and it doesn’t work for me. Am I doing something wrong?

npx cypress-tags run -e TAGS=/“@desktop and @tag2/“

[rkrisztian]

@jime2003 , I don't think I can help because I never used cypress-tags because it's not easy to directly use the command cypress in an nrwl/nx project, and npx nx e2e --env TAGS='...' does not work there. However, I could specify the TAGS env vars in cypress.json or by typing CYPRESS_TAGS='...' npx nx e2e.

[RitikaGhosh15]

Hello @rkrisztian, Pls follow the below steps to reproduce the scenario Feature file: Feature: Demo Testcase

Feature Description

@desktop
Scenario Outline:
    Given Open page  with "<link>"
    @tag1
    Examples:
        | link     | 
        | facebook | 

    @tag2
    Examples:
        | link     | 
        | linkedin | 

Step defination file: import { Given } from 'cypress-cucumber-preprocessor/steps'; Given('Open page with {string}', (param) => { cy.viewport(1440, 900); cy.visit(https://www.${param}.com/); cy.get('title').should('exist'); });

Command: npx cypress open --env env=devint --env TAGS='@desktop and @tag2'

package.json: "cypress": "^8.0.0", "cypress-cucumber-preprocessor": "^4.2.0",

video: https://user-images.githubusercontent.com/82997275/128142613-e5c5c140-2f9d-45e9-a78c-502c4510c1b8.mp4

[rkrisztian]

@RitikaGhosh15 , I don't know what's going on there, but I don't think any tags are filtered at all.

[RitikaGhosh15]

@rkrisztian unfortunately example tags are not getting filtered.

[rkrisztian]

@RitikaGhosh15 what makes you think that only example tags are not filtered? From what I can see, your problem could be unrelated to this issue.

[RitikaGhosh15]

@rkrisztian i have segregated the scenario outline for desktop and mobile, its able to differentiate between them but not able to differentiate between examples tag. command: npx cypress open --env env=devint --env TAGS='@mobile and @tag2'

[hari49951]

@RitikaGhosh15 It's working for me as expected. why are you using both tags --env TAGS='@mobile and @tag2'. I think you should use --env TAGS='@tag2' it will run the scenario with example which has "tag2".

[RitikaGhosh15]

@hari49951 but if i use only example tag i.e @tag2, i wont able to select my breakpoint i.e mobile, desktop or tablet which i have declared at top. am using the above tag i.e @mobile to differentiate between the breakpoints.

Now its running for both the breakpoint for tag2 but that's not what's expected.

[rkrisztian]

@hari49951 it's wrong to tell what tags to use. @RitikaGhosh15 look, we will never be able to reproduce because we don't have enough info, like an example project.

[RitikaGhosh15]

@rkrisztian ohk but i was expecting it to resolve under this issue...thanks for your time!

[dankopetrovic]

This is not working for me as well. I have tried it with the samples from this repository. This feature file was newly added, and it doesn't filter the tag set on Examples: https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/blob/master/cypress/integration/TagsImplementationWithEnvSetExamplesLevel.feature

package.json "cypress": "^8.2.0", "cypress-cucumber-preprocessor": "^4.2.0",

npx cypress-tags run -e TAGS='@test-tag'
Using cypress.json configuration:
Spec files:  cypress/integration/**/*.feature

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    8.2.0                                                                              │
  │ Browser:    Electron 89 (headless)                                                             │
  │ Specs:      2 found (TagsImplementationWithEnvSet.feature, TagsImplementationWithEnvSetScenari │
  │             oLevel.feature)                                                                    │
  │ Searched:   cypress/integration/TagsImplementationWithEnvSet.feature, cypress/integration/Tags │
  │             ImplementationWithEnvSetScenarioLevel.feature                                      │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

(Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  TagsImplementationWithEnvSet.featur      113ms        3        3        -        -        - │
  │    e                                                                                           │
  ├────────────────────────────────────────────────────────────────────────────────────────────────┤
  │ ✔  TagsImplementationWithEnvSetScenari       80ms        2        1        -        1        - │
  │    oLevel.feature                                                                              │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        193ms        5        4        -        1        -  

[rkrisztian]

@dankopetrovic , OK, I tried https://github.com/TheBrainFamily/cypress-cucumber-example, although it seems a bit too old. Regardless, after updating the preprocessor to the latest version, I tried to check if we can tag examples individually there and filter for those tags:

• CYPRESS_TAGS='@tagS and @tagE2' npm run test => Worked.
• npx cypress run -e TAGS='@tagS and @tagE2' => Worked.

What did not work:

$ DEBUG=true npx cypress-tags run -e TAGS='@tagS and @tagE2'
Using cypress.json configuration:
Spec files:  {cypress/integration/**/*.{feature,cypress/integration/features}}
Ignored files:  *.js
DEBUG: Found glob [ '{cypress/integration/**/*.{feature,cypress/integration/features}}' ]
DEBUG: Found tag expression [ '@tagS and @tagE2' ]
No matching tags found

=> I think the problem is with the cypress-tags script, which seems to fail to find tagged examples (it works for TAGS='@tagS), but your reported errors are different from mine. While I think it makes sense to look into the cypress-tags script and see why I got this error (which I will try), I think it would be still useful to see an example project that lets me reproduce the exact same issue you guys reported, just to be sure we are dealing with the same error.

Edit: I found the problem:

      let testFiles = !Array.isArray(cypressConf.testFiles)
        ? cypressConf.testFiles.split(",")
        : cypressConf.testFiles;

This is just plain wrong when cypressConf.testFiles is set to **/*.{feature,features}. After doing a workaround, I still get a similar error:

No matching tags found

And that is because the tag filtering script does not take examples into account. This one should be an easy fix.

[rkrisztian]

Opened PR https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/pull/607 for the feature files search bug.

I can also work on the examples tag filtering bug if you want...

[dankopetrovic]

@rkrisztian Thanks for looking into this issue. I would really appreciate if you could work on examples tag filtering...

I've set-up an example project (clone from https://github.com/TheBrainFamily/cypress-cucumber-example, with addition of examples with tags): https://github.com/dankopetrovic/cypress-cucumber-example (credits to @RitikaGhosh15)

@features
Feature: Feature Description

    @desktop
    Scenario Outline: Opens a link <link>
        Given Open page with "<link>"

        @tag1
        Examples:
            | link     |
            | facebook |
            | twitter  |

        @tag2
        Examples:
            | link     |
            | linkedin |

npx cypress-tags run -e TAGS='@tag2'
Using cypress.json configuration:
Spec files:  cypress/integration/**/*.feature
Ignored files:  *.js
No matching tags found

[RitikaGhosh15]

Thank you @rkrisztian @dankopetrovic for looking into this issue.

[jime2003]

@dankopetrovic , OK, I tried https://github.com/TheBrainFamily/cypress-cucumber-example, although it seems a bit too old. Regardless, after updating the preprocessor to the latest version, I tried to check if we can tag examples individually there and filter for those tags:

• CYPRESS_TAGS='@tagS and @tagE2' npm run test => Worked.
• npx cypress run -e TAGS='@tagS and @tagE2' => Worked.

What did not work:

$ DEBUG=true npx cypress-tags run -e TAGS='@tagS and @tagE2'
Using cypress.json configuration:
Spec files:  {cypress/integration/**/*.{feature,cypress/integration/features}}
Ignored files:  *.js
DEBUG: Found glob [ '{cypress/integration/**/*.{feature,cypress/integration/features}}' ]
DEBUG: Found tag expression [ '@tagS and @tagE2' ]
No matching tags found

=> I think the problem is with the cypress-tags script, which seems to fail to find tagged examples (it works for TAGS='@tagS), but your reported errors are different from mine. While I think it makes sense to look into the cypress-tags script and see why I got this error (which I will try), I think it would be still useful to see an example project that lets me reproduce the exact same issue you guys reported, just to be sure we are dealing with the same error.

Edit: I found the problem:

      let testFiles = !Array.isArray(cypressConf.testFiles)
        ? cypressConf.testFiles.split(",")
        : cypressConf.testFiles;

This is just plain wrong when cypressConf.testFiles is set to **/*.{feature,features}. After doing a workaround, I still get a similar error:

No matching tags found

And that is because the tag filtering script does not take examples into account. This one should be an easy fix.

@rkrisztian Thanks for looking into this issue. I would really appreciate if you could work on examples tag filtering...

I've set-up an example project (clone from https://github.com/TheBrainFamily/cypress-cucumber-example, with addition of examples with tags): https://github.com/dankopetrovic/cypress-cucumber-example (credits to @RitikaGhosh15)

@features
Feature: Feature Description

    @desktop
    Scenario Outline: Opens a link <link>
        Given Open page with "<link>"

        @tag1
        Examples:
            | link     |
            | facebook |
            | twitter  |

        @tag2
        Examples:
            | link     |
            | linkedin |

npx cypress-tags run -e TAGS='@tag2'
Using cypress.json configuration:
Spec files:  cypress/integration/**/*.feature
Ignored files:  *.js
No matching tags found

Hopefully, below command would work where I am trying to switch environment at run time specify in cypress.json. It is not working right now. npx cypress-tags run -e TAGS='@tag1 and @tag2' --env env=qa --browser chrome --headed"

[rkrisztian]

@RitikaGhosh15 , @dankopetrovic , PR https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/pull/609 opened.

@jime2003 , I think the env problem should be a separate issue report to keep things focused.

[jime2003]

@RitikaGhosh15 , @dankopetrovic , PR #609 opened.

@jime2003 , I think the env problem should be a separate issue report to keep things focused.

thanks @rkrisztian . I have opened new issue #610

[lgandecki]

:tada: This issue has been resolved in version 4.3.0 :tada:

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot :package::rocket:

[majco333]

It's working but screenshots are generated for other scenarious in the same feature file (without tag)...

Cypress version: 8.5.0: Preprocessor version: 4.3.0:

[badeball]

@majco333, possible fixed by https://github.com/badeball/cypress-cucumber-preprocessor/pull/4. Can you try

$ npm install "git://github.com/badeball/cypress-cucumber-preprocessor#ensure-unique-outline-names"

... and see if it helps?