cypress-io / cypress-grep

Filter tests using substring
137 stars 19 forks source link

How can I repeatedly run/burn a scenario (written in cucumber) for a specific number of times through cypress-grep? #62

Closed praveenpandey02 closed 3 years ago

praveenpandey02 commented 3 years ago

I have installed cypress-grep plugin as a dev dependency so that I can run a cucumber scenario a specific number of times to check its flakiness. These are the changes I have made:

  1. yarn add -D cypress-grep which adds this dependency in package.json
  2. Made this entry require('cypress-grep')() in cypress/support > index.js
  3. Optionally, also made the following below entry here cypress/plugins > index.js
    module.exports = (on, config) => {
    require('cypress-grep/src/plugin')(config)
    }
  4. Now I am trying to run a scenario (title contains "hello") inside a feature file multiple times to check its flakiness cypress/integration/folderA/sample.feature. For this, I am executing the following command: npx cypress run --spec "cypress/integration/folderA/sample.feature" --env grep=hello,burn=10

But every time the command runs, I get the following error:

The error was:

Error: Webpack Compilation Error
./cypress/integration/folderA/sample.feature 1:15
Module parse failed: Unexpected token (1:15)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> Feature: Feature file name
| 
|   Background: Sample background title

This occurred while Cypress was compiling and bundling your test code. This is usually caused by:

- A missing file or dependency
- A syntax error in the file or one of its dependencies

Fix the error in your code and re-run your tests.

I am not sure what this error is trying to tell me and how do I fix it. P.S: I have even referred this but it only talks about spec (.js) files and not feature (.feature) files: https://glebbahmutov.com/blog/burning-tests/

bahmutov commented 3 years ago

Since I don't use cucumber, have you set up the cucumber preprocessor that loads the .feature files in Cypress?

praveenpandey02 commented 3 years ago

Since I don't use cucumber, have you set up the cucumber preprocessor that loads the .feature files in Cypress?

Yes I've set it up and my feature files otherwise run perfectly fine.

bahmutov commented 3 years ago

ok, in that case can you provide a reproduction repo with cucumber working but then adding cypress-grep and it stops working?

praveenpandey02 commented 3 years ago

Sure I can do that.

praveenpandey02 commented 3 years ago

Sorry. It took me sometime to get back to you. Here is the repo link: https://github.com/praveenpandey02/plug_it. I had to create a new repo due to data sensitivity issue but the error I get is exactly the same. This repo was tested both with cypress-grep plugin and without it. Unfortunately, it does not work with cypress-grep. The repo does not have the plugin installed so that when you clone it, you can see the cucumber feature files running smoothly. When the plugin is installed, additional code to support the plugin are already commented out in Plugins index.js and Support index.js (as per this page: https://github.com/cypress-io/cypress-grep)

praveenpandey02 commented 3 years ago

Hi @bahmutov. We are looking forward to your suggestion here.

praveenpandey02 commented 3 years ago

Still no luck! It works with spec files and not with feature files :(

praveenpandey02 commented 3 years ago

So, I finally made it work and what a silly mistake I've been making all this while 🤦‍♂️. So this was wrong:

image

Instead, I had to do this:

image

This is the result. All green:

image
bahmutov commented 3 years ago

👍

Sent from my iPhone

On Aug 25, 2021, at 07:37, Praveen @.***> wrote:

 Closed #62.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.