Tags aren't working

[jakub-kolosowski-pirum]

Hi! We have encountered that even though you have unit tests for tags/filtering we faced the problem that it runs all of them every time. Version installed: 4.1.2 We followed documentation so put custom tag before Scenario -> @ignore and then we added in describeFeature excludeTags and all the tests are still being triggered. We tried also using configuration - setVitestCucumberConfiguration but it is not working either. Is it a bug or we are missing something that's not included in docs?

[amiceli]

Hi, @ignore is already use in default option, you don't have to add it.

Do you have excluded other tags ? When you run test, you have error like "Scenario test is not called" ?

[jakub-kolosowski-pirum]

I did have used other tags to exclude than @ignore. Answering your second question, no I don't have that error, just all tests are passing like running without tags.

[amiceli]

Can you show some code please. I think I found a bug, can you test to add "ignore" in excludeTags where you added other tag.

Example, replace excludeTags: ["ignore-e2e"] with excludeTags: ["ignore-e2e", "ignore"].

[jakub-kolosowski-pirum]

So I have added @ignore tag to my scenario:

Feature: XYZ feature

  @ignore
  Scenario: XYZ scenario
    Given XYZ given
    When XYZ when
    Then XYZ then
    And XYZ and

and put in describe feature in steps

const feature = await loadFeature("features/index.feature");

describeFeature(
  feature,
  ({ Scenario }) => {
    Scenario(
      "XYZ scenario",
      ({ Given, When, Then, And }) => {
        Given(
          "XYZ given",
          () => {}
        );

        When("XYZ when", () => { });

        Then("XYZ then", () => {});

        And("XYZ and", () => {});
      });
    },
    { excludeTags: ["ignore", "ignore-e2e"] }
  );

still the test is being run

[jakub-kolosowski-pirum]

On top of that when I change ignore to something different for example like in docs awesome it's still running that test and passes

[amiceli]

Ok, I'm going to check this and I'll made a fix.

[amiceli]

Ok I understand problem ;) If you use @ignore in a Scenario/Rule/Background etc, you shouldn't call it in spec file.

I can make a fix but I will throw an error if you call Scenario() for an excluded scenario (same for Rule, Background etc).

[Odonno]

I tried that with the Playground we have in the docs website. Adding the @ignore tag indeed does nothing.

@amiceli It should not throw an error but execute describe.skip if an exclusion tag is present.

[amiceli]

This is a bug, I will fix it on Playground too. No, when you ignore a Scenario, it shouldn't be implemented. It's like keep useless code in a project.

I will throw en error in fix but you can remove @ignore tag and use keep skip in step if you want to keep scenario test code.

An another idea, I can implement a @skip tag, to skip Scenario/Rule/Background tests.

[Odonno]

No, it should not throw. This is not what developers would expect. The idea is to be able to avoid running some Scenarii. But code should not be removed because it would require too much effort just to run the tests without some Scenarii. See the original implementation and documentation here: https://github.com/bencompton/jest-cucumber/blob/main/docs/AdditionalConfiguration.md#tag-filtering

However, if you want to have a proper naming convention, one that does not give confusion, then that could be an idea.

But then, I don't see the need of excludeTags as it would bring confusion with this new idea. That would mean to do:

• rename excludeTags by skipTags and then have a default value of @skip (instead of the current @ignore)
• if we go that way, why not also todoTags with a default value of @todo to mark unfinished scenarii (and steps?)

[amiceli]

Ok now I understand. I'm ok to call skip when a scenario should be ignored ;). I will make a fix for this.

[francesco-albanese]

I think that excludeTags and includeTags would still be very useful, especially when the same batch of end to end tests can be executed differently based on requirements. For example the tag @smoke can be used only to run scenarios tagged with @smoke , by a CI to do regression tests. And the tags should be specified as an environment variable so that can be controlled from a CI environment.

[amiceli]

@francesco-albanese For env variable you'll see something like that ?

• process.env.VITEST_INCLUDE_TAGS
• process.env.VITEST_EXCLUDE_TAGS

And vitest-cucumber will use them in options before run tests.

[francesco-albanese]

Yes that would work very well. But most important thing is being able to support scenario filtering based by tags, just like jest cucumber and similar libraries. Using tags in features files is a common practice for scenario filtering and it's even described in the official cucumber documentation https://cucumber.io/docs/cucumber/api/?lang=javascript

[amiceli]

Ok for env variable, I will work on another PR for it. For tags currently vitest-cucumber provides includeTags and excludeTags.

On the fix I will call skip for scenarios not matching includeTags.

[amiceli]

I've published a beta version 4.1.3-beta.1. You can test it. I'm making some tests with large projects.

[jakub-kolosowski-pirum]

Amazing @amiceli! It works properly with any tags now for exclude and include also in configuration. Thank you so much!

[amiceli]

Nice, thanks for your tests ;). I'm going to publish a stable version.

[amiceli]

I published a stable version 4.1.3. I will update doc about tags, skip etc.

@francesco-albanese I will open another issue for env variables.