cypress-io / cypress-grep

Filter tests using substring
137 stars 19 forks source link

Can't run grepFilterSpecs and grepOmitFiltered #87

Closed darioprinc closed 2 years ago

darioprinc commented 2 years ago

Hi, I am struggling to run the grepFilterSpecs and grepOmitFiltered options.

Can't get it to work, not in CI, but mainly not even locally.

plugins/index.js

config.env.grepTags = '@smoke';
config.env.grepFilterSpecs = true;
config.env.grepOmitFiltered = true;

or

CLI

npx cypress run --env grepTags=@smoke,grepFilterSpecs=true,grepOmitFiltered=true

or

cypress.json

{
  "env": {
    "grepTags": "@smoke",
    "grepFilterSpecs": true,
    "grepOmitFiltered": true
  }
}

What is interesting - the configurations above will correctly run only tests with a @smoke tag, but the grepFilterSpecs and grepOmitFiltered are not taken into account. The output still shows pending tests both in run mode and in GUI:

image

Expected: Tests without the @smoke tag should not be skipped (pending) but ommited.

The plugin is registered in plugins/index.js require('cypress-grep/src/plugin')(config); Cypress v. 8.7.0

Any hint will be appreciated.

Cheers, Dario

bahmutov commented 2 years ago

Can you paste the full plugins file? Do you return the config file? https://github.com/cypress-io/cypress-grep#plugin-file

// cypress/plugins/index.js
module.exports = (on, config) => {
  // optional: register cypress-grep plugin code
  // https://github.com/cypress-io/cypress-grep
  require('cypress-grep/src/plugin')(config)
  // make sure to return the config object
  // as it might have been modified by the plugin
  return config
}
darioprinc commented 2 years ago

Sure:

import { prepareTags } from '../support/grepTags/prepareTags';

module.exports = async (on, config) => {
  require('cypress-grep/src/plugin')(config);

  const location = config.env.location || 'develop';

  config.env = require(`../config/${location}.json`);

  config.retries = config.env.retries;

  config.baseUrl = config.env.baseUrl;

  // runs only in CI
  if (process.env.CI_MERGE_REQUEST_DIFF_BASE_SHA && process.env.CI_COMMIT_SHA) {
    config.env.grepTags = await prepareTags(); // this returns a string, e.g. '@react @smoke'
    config.env.grepFilterSpecs = true;
    config.env.grepOmitFiltered = true;
  }

  return config;
};
Gayathri-Kesavakannan commented 2 years ago

Facing same issue , filtering using tag “smoke” filtering specs using tag “smoke” Couldn’t determine file

cypress run grepTags=smoke is working fine runs only smoke test but moves other tests to Pending

cypress run grepTags=smoke,grepFilterSpecs=true - this is not working. Couldn’t determine test names in file

not sure what am missing here

bahmutov commented 2 years ago

Please look at the debugging section of the readme

Sent from my iPhone

On Nov 11, 2021, at 07:29, Gayathri-Kesavakannan @.***> wrote:

 Facing same issue , filtering using tag “smoke” filtering specs using tag “smoke” Couldn’t determine file

cypress run grepTags=smoke is working fine runs only smoke test but moves other tests to Pending

cypress run grepTags=smoke,grepFilterSpecs=true - this is not working. Couldn’t determine test names in file

not sure what am missing here

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

darioprinc commented 2 years ago

Yep, everything in place:

image

The tests run by tag but the rest is still skipped:

image

bahmutov commented 2 years ago

Can you. show the debug output in the browser? https://github.com/cypress-io/cypress-grep#debugging-in-the-browser

darioprinc commented 2 years ago

Can't see the options there:

image

bahmutov commented 2 years ago

Can you check the plugin version you are using? https://github.com/cypress-io/cypress-grep#debugging

darioprinc commented 2 years ago

well, version upgrade solved it. thanks a lot!

dkoudrinhc commented 2 years ago

I'm having a similar issue where it IS running my tagged tests, but the omitFiltered=true flag isn't being picked up for some reason.

darioprinc commented 2 years ago

did you follow the steps above? register in plugins, upgrade package to the latest version etc? you got the flag name right - grepOmitFiltered ?

dkoudrinhc commented 2 years ago

Yep, I think I got it working after a day of trial and error.

EvgeniyGordinskiy commented 2 years ago

grepOmitFiltered doesn't work. I've checked all possible solutions above: registered in plugins, installed the latest version (2.13.1), checked the flag names.

image