cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.74k stars 3.16k forks source link

@cypress/grep All the specs are listed if there is no matching tag #25020

Open ajil-paul opened 1 year ago

ajil-paul commented 1 year ago

Current behavior

Consider there are only 3 specs in the cypress. And 3 of them have @smoke as the tag. But If I run the cypress with grepTags = @fast Then cypress is listing all the 3 specs. But those specs have only a @smoke label.

Let's say, I have added a @fast tag for one spec. Then it will list only that matching spec.

So if there are no specs that have the matching tag, then cypress will list all the specs.

Desired behavior

If there are no specs with the tag @fast and we run with grepTags = @fast, then Cypress should show an empty list of specs instead of listing all specs.

Test code to reproduce

env: {
    grepFilterSpecs: true,
    grepOmitFiltered: true,
    grepTags: "@fast"
  },

Add some specs with @smoke as tags and run.

Cypress Version

11.2

Node version

v18.12.1

Operating System

macOs

Debug Logs

No response

Other

No response

goatfryed-holos commented 1 year ago

This behavior is a pain for new developers. I started using cypress/grep today to replace some custom tag filter logic. I was under the impression that the setup wasn't working at all, because my tag filter never skipped anything.

In the end, it was just a typo in the tag, so of course it wasn't matching. But that argument didn't do anything and every grepTags=I_DONT_EXIST didn't do anything either. So I spent an hour looking for whether I did something wrong in the setup.

testA113 commented 6 months ago

Would love a fix on this too. If we're running this on CI, then it burns through resources without us immediately noticing. In the case of no matching specs, it would be great to exit early, with a sensible warning message