webdriverio-boneyard / wdio-cucumber-framework

A WebdriverIO v4 plugin. Adapter for Cucumber testing framework.
MIT License
77 stars 61 forks source link

tagExpression doesn't allow to add more than one tag. #80

Open arunaousula9421 opened 7 years ago

arunaousula9421 commented 7 years ago

Hi, Below is my CucumberOptns where I am passing the tags to run specific tests

cucumberOpts: {
        require: ['./src/step_definitions/loginSteps.ts', './src/step_definitions/searchSteps.ts', './src/support/hook.js'],        // <string[]> (file/dir) require files before executing features
        backtrace: false,   // <boolean> show full backtrace for errors
        compiler: ["ts:ts-node/register"],       // <string[]> ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable)
        dryRun: false,      // <boolean> invoke formatters without executing steps
        failFast: false,    // <boolean> abort the run on first failure
        format: ['pretty'], // <string[]> (type[:path]) specify the output format, optionally supply PATH to redirect formatter output (repeatable)
        colors: true,       // <boolean> disable colors in formatter output
        snippets: true,     // <boolean> hide step definition snippets for pending steps
        source: true,       // <boolean> hide source uris
        profile: [],        // <string[]> (name) specify the profile to use
        strict: false,      // <boolean> fail if there are any undefined or pending steps
        tags: [],
        tagExpression: '@login,@search',           // <string[]> (expression) only execute the features or scenarios with tags matching the expression
        timeout: 20000,     // <number> timeout for step definitions
        ignoreUndefinedDefinitions: true, // <boolean> Enable this config to treat undefined definitions as warnings.
    },

For Example, I have two feature files with tags @login and @search. If I add only one tag which is @login it spins up two instances where in one instance it doesn't run the tests and in the second instance it runs the tests for the @login. Actually it should spawn up only one instance. As I have two features it spawn 2 instances.

In the other situation as above in the cucumberOpts, if I pass two tags it spawn two instances and didn't run both the tests. Skipped both of them.

Tried adding different combinations of tag expression. For example: if I add @login not @search, it does same as above open two instances and run one test[@login] and in another instance launches browser and doesn't run anything[@search].

And if I add @login and search, it launches instances for both and skips the tests.

Can I please know if this how it behaves.

glens-games commented 7 years ago

Have you tried using '@login or @search' yet?

I had to go through this recently. There's a small snippet in their changelog under 2.0.0-rc.0 that explains the new format for their tag expressions.

arunaousula9421 commented 7 years ago

@glenheide I have done all the things as they explained. But when I say tagExpression as '@login and @search' it skips both the features but launches the instances and doesn't run any tests. Similarly with '@login or @search'. I am using v1.0.2 of wdio-cucumber-framework

markmssd commented 6 years ago

I am having the same issue... @arunaousula9421 have you found a solution?

BorisOsipov commented 6 years ago

@markmssd you can use workaround

in your wdio.conf you can manually filter specs with your tags like that:

const fs = require('fs');
const { TagExpressionParser } = require('cucumber-tag-expressions');
const glob = require('glob');

const featuresPath = './src/features/**/*.feature';
const tagParser = new TagExpressionParser();

const featureFilesWithTags = (featuresPath, cucumberTags) => {
  const expressionNode = tagParser.parse(cucumberTags);
  const featureFilesHaveTags = glob.sync(featuresPath).filter((featureFile) => {
    const content = fs.readFileSync(featureFile, 'utf8');
    if (content.length > 0) {
      const tagsInFile = content.match(/(@\w+)/g) || [];
      if (expressionNode.evaluate(tagsInFile)) {
        return true;
      }
    }
    return false;
  });

  return featureFilesHaveTags;
};

exports.config = {
....
  specs: featureFilesWithTags(featuresPath, '@login or @search')),
  cucumberOpts: {
   tagExpression: '@login or @search'),
....
}
....
}
markmssd commented 6 years ago

It works, thanks a lot @BorisOsipov! 🎉

BorisOsipov commented 6 years ago

@markmssd you're welcome. pay attention it's a bit raw and it handles emails in your feature files as tag (because emails have @ symbol)

markmssd commented 6 years ago

Right, if we use the regex content.match(/(^@\w+)/g) instead (note the ^), it should handle tags only

BorisOsipov commented 6 years ago

@markmssd yep :)

venkatteshb commented 6 years ago

const fs = require('fs'); const { TagExpressionParser } = require('cucumber-tag-expressions'); const glob = require('glob'); const featuresPath = './src/features/*/.feature'; const tagParser = new TagExpressionParser(); const featureFilesWithTags = (featuresPath, cucumberTags) => { const expressionNode = tagParser.parse(cucumberTags); const featureFilesHaveTags = glob.sync(featuresPath).filter((featureFile) => { const content = fs.readFileSync(featureFile, 'utf8'); if (content.length > 0) { const tagsInFile = content.match(/(@\w+)/g) || []; if (expressionNode.evaluate(tagsInFile)) { return true; } } return false; }); return featureFilesHaveTags; }; exports.config = { .... specs: featureFilesWithTags(featuresPath, '@login or @search')), cucumberOpts: { tagExpression: '@login or @search'), .... } .... }

Should this code go in wdio.conf.js ? @BorisOsipov

BorisOsipov commented 6 years ago

yes

чт, 7 июн. 2018 г. в 22:39, Venkat Rao notifications@github.com:

const fs = require('fs'); const { TagExpressionParser } = require('cucumber-tag-expressions'); const glob = require('glob');

const featuresPath = './src/features/*/.feature'; const tagParser = new TagExpressionParser();

const featureFilesWithTags = (featuresPath, cucumberTags) => { const expressionNode = tagParser.parse(cucumberTags); const featureFilesHaveTags = glob.sync(featuresPath).filter((featureFile) => { const content = fs.readFileSync(featureFile, 'utf8'); if (content.length > 0) { const tagsInFile = content.match(/(@\w+)/g) || []; if (expressionNode.evaluate(tagsInFile)) { return true; } } return false; });

return featureFilesHaveTags; };

exports.config = { .... specs: featureFilesWithTags(featuresPath, '@login or @search')), cucumberOpts: { tagExpression: '@login or @search'), .... } .... }

Should this code go in wdio.conf.js ? @BorisOsipov https://github.com/BorisOsipov

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webdriverio/wdio-cucumber-framework/issues/80#issuecomment-395540385, or mute the thread https://github.com/notifications/unsubscribe-auth/AGNmZA65Z5JohITHH0Z4JHE9IMsL-tQBks5t6YFbgaJpZM4QKDUy .

--

-- Best Regards, Boris Osipov

id4automation commented 6 years ago

@BorisOsipov thanks for the snippet. It works , but it opens multiple browser instances (equal to the number of features file). Only one feature file has the tag I intend to run. Is there a way to restrict the browser instance to open only one?

BorisOsipov commented 6 years ago

but it opens multiple browser instances

don't know why. it must not. for me it works like a charm

ckurban commented 5 years ago

@BorisOsipov In earlier version of the wdio-cucumber-framework I used to use below written line : let cucumberTags = browser.options.cucumberOpts.tagExpression.filter((tags) => {return tags.startsWith('@')});

After upgrading my wdio-cucumber-framework to version 2.2.8 - filter does not work, is there any other function such as filter that I can use browser.options.cucumberOpts.tagExpression.????

Any suggestions ?

BorisOsipov commented 5 years ago

Use cucumber-tag-expressions for properly parcing tagExpression.

ckurban commented 5 years ago

@BorisOsipov I used cucumber-tag-expressions and it is failing when I have:

tagExpression: '@uk and not @defect',

with output :

evaluate=function (variables) {
      return leftExpr.evaluate(variables) && rightExpr.evaluate(variables)
    }, toString=function () {
      return '( ' + leftExpr.toString() + ' and ' + rightExpr.toString() + ' )'
    }
ckurban commented 5 years ago

It seems to not able to handle and and not operators. Is there any way to skip the negate condition such asnotas some of my scenarios have@defect` on them ?

BorisOsipov commented 5 years ago

It seems to not able to handle and and not operators. I used cucumber-tag-expressions and it is failing when I have:

how do you use it?

BorisOsipov commented 5 years ago

It works. Cucumber uses cucumber-tag-expressions internally for working with tags.

ckurban commented 5 years ago

@BorisOsipov Here how I use it :

        const tagParser = new TagExpressionParser();

       logger.info(' ===== EXPRESSION  =====', browser.options.cucumberOpts.tagExpression);
        const expressionNode = tagParser.parse(browser.options.cucumberOpts.tagExpression);
        logger.info(' ===== TAGS  =====', expressionNode);
        const featureFilesHaveTags = glob.sync(featuresPath).filter((featureFile) => {
            const content = fs.readFileSync(featureFile, 'utf8');
            if (content.length > 0) {
                const tagsInFile = content.match(/(@\w+)/g) || [];
                if (expressionNode.evaluate(tagsInFile)) {
                    return true;
                }
            }
            return false;
        });

Here is the output :

info:  ===== EXPRESSION  ===== @uk and not @defect
info:  ===== TAGS  ===== evaluate=function (variables) {
      return leftExpr.evaluate(variables) && rightExpr.evaluate(variables)
    }, toString=function () {
      return '( ' + leftExpr.toString() + ' and ' + rightExpr.toString() + ' )'
    }

Here is my cucumberOpts:

 cucumberOpts: {
        require: ['./test/step_definitions/**/*.js',],   // <string[]> (file/dir) require files before executing features
        backtrace: true,    // <boolean> show full backtrace for errors
        compiler: [],       // <string[]> ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable)
        dryRun: false,      // <boolean> invoke formatters without executing steps
        failFast: true,    // <boolean> abort the run on first failure
        format: ['pretty'], // <string[]> (type[:path]) specify the output format, optionally supply PATH to redirect formatter output (repeatable)
        colors: true,       // <boolean> disable colors in formatter output
        snippets: false,    // <boolean> hide step definition snippets for pending steps
        source: false,      // <boolean> hide source uris
        profile: [],        // <string[]> (name) specify the profile to use
        strict: false,      // <boolean> fail if there are any undefined or pending steps
        tagsInTitle: false,  // <boolean> add cucumber tags to feature or scenario name
        timeout: 60000,     // <number> timeout for step definitions
        tagExpression: '@uk and not @defect',
        ignoreUndefinedDefinitions: true, // <boolean> Enable this config to treat undefined definitions as warnings.
    },
BorisOsipov commented 5 years ago

And why did you decide that is not work? You just print ' ===== TAGS =====', expressionNode); in console...

ckurban commented 5 years ago

Well, I am trying to read all tags and assign the right value to my next statement:

  cucumberTags.forEach((tag) => {
            switch (tag.toString()) {
                case '@spb':
                    userCategory = constants.userCategory.spb;
                    break;

                case '@sk':
                    userCategory = constants.userCategory.sk;
                    break;

                case '@uk':
                    userCategory = constants.userCategory.uk;
                    break;

                case '@fr':
                    userCategory = constants.userCategory.fr;
                    break;

                case '@it':
                    userCategory = constants.userCategory.it;
                    break;
            }
        });
BorisOsipov commented 5 years ago
const expressionNode = tagParser.parse(browser.options.cucumberOpts.tagExpression);

if(expressionNode.evaluate('@spb')) {
    userCategory = constants.userCategory.spb;
} else if(@expressionNode.evaluate('@sk')){
    userCategory = constants.userCategory.sk;
}
.....

smth like this?

ckurban commented 5 years ago

Works like a charm !!!!! Thanks @BorisOsipov

ckurban commented 5 years ago

@BorisOsipov me again, I know it is not related to this thread and if needed i will create separate issue but when I am trying to set my test data to global in my before hook and it seems, it does not read the json file - all other .js files are OK.

Here is my code :

    before: function () {

        i18n.configure({
            directory: './test/config/locales',
            locales: ['en'],
            fallbacks: {'fr': 'en'},
            defaultLocale: 'en'
        });

        global.topMenu = require('../../../../utils/topMenu');
        global.envConfig = require('./testData.json');
        global.constants = require('../../../../utils/constants');
        global.helpers = helpers;
        global.expect = chai.expect;
        global.should = chai.should();
        global.userCategory = user;
        global.scenarioTags = tags;
        global.scenarioTag = tag;
        global.logger = require('winston');
        global.i18n = i18n;
    }

Any help appreciated ?

BorisOsipov commented 5 years ago

@ckurban please when you write "seems it doesn't work", "it seems, it does not read" in issues explain what you expect and what happened then. It is hard to guess what do you mean by "it does not read", any errors\failed checks?

ckurban commented 5 years ago

@BorisOsipov I am trying to set test data to global.envconfig object and when I am trying to use any data from envconfig such as envConfig.url.webservicesEndpoint it throws ERROR: Cannot read property 'url' of undefined

It was working before my upgrade where I was able to retrieve the url and endpoints from the global envConfig object.

Does that make sense ?

BorisOsipov commented 5 years ago

@ckurban Not sure what is going on but I guess this will help

const testData = require('./testData.json');

export.config = {

global.envConfig = testData;
}
ckurban commented 5 years ago

@BorisOsipov I am keep getting this error:

**ERROR: envConfig is not defined**
chrome.68_0
Reference    at Object.<anonymous> (/Users/xxx/dev/xxx/test/api/xxxx/ws410.js:7:57)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Module._load (module.js:498:3)
    at Function.hookedLoader [as _load] (/Users/xxx/dev/xxx/node_modules/mockery/mockery.js:111:12)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/xxx/dev/xxx/test/step_definitions/delivery_steps.js:3:15)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Module._load (module.js:498:3)
    at Function.hookedLoader [as _load] (/Users/xxx/dev/xxx/node_modules/mockery/mockery.js:111:12)
    at Module.require (module.js:597:17)
    at require (internal/module.js:11:18)
    at /Users/xxx/dev/xxx/node_modules/wdio-cucumber-framework/build/adapter.js:259:17
    at Array.forEach (<anonymous>)
    at CucumberAdapter.loadSpecFiles (/Users/xxx/dev/xxx/node_modules/wdio-cucumber-framework/build/adapter.js:250:34)
    at CucumberAdapter._callee$ (/Users/xxx/dev/xxx/node_modules/wdio-cucumber-framework/build/adapter.js:135:38)
    at tryCatch (/Users/xxx/dev/xxx/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/Users/xxx/dev/xxx/node_modules/regenerator-runtime/runtime.js:296:22)
    at Generator.prototype.(anonymous function) [as next] (/Users/xxx/dev/xxx/node_modules/regenerator-runtime/runtime.js:114:21)
    at step (/Users/xxx/dev/xxx/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /Users/xxx/dev/xxx/node_modules/babel-runtime/helpers/asyncToGenerator.js:35:14
    at new Promise (<anonymous>)
    at new F (/Users/xxx/dev/xxx/node_modules/core-js/library/modules/_export.js:35:28)
    at CucumberAdapter.<anonymous> (/Users/xxx/dev/xxx/node_modules/babel-runtime/helpers/asyncToGenerator.js:14:12)
    at CucumberAdapter.run (/Users/xxx/dev/xxx/node_modules/wdio-cucumber-framework/build/adapter.js:214:29)
    at Object._callee2$ (/Users/xxx/dev/xxx/node_modules/wdio-cucumber-framework/build/adapter.js:347:40)
    at tryCatch (/Users/xxx/dev/xxx/node_modules/regenerator-runtime/runtime.js:62:40)
    at Generator.invoke [as _invoke] (/Users/xxx/dev/xxx/node_modules/regenerator-runtime/runtime.js:296:22)

hooks such as before does not work in latest wdio-cucumber-adapter as it is not setting values into global variable.