[Feature] BDD implementation with Playwright test runner

[ShanmukhaGajula]

Our team has a case to use the BDD test, Playwright has any plan to add their own BDD implementation using the playwright test runner? That would be great if to have.

This can help many people who want to use BDD along with Playwright. That's where cucumber became popular 😉

[PavanMudigonda]

I am also in the same situation, not able to ask teams ( I am referring to hundreds of teams basically the whole company I work at) to use Playwright as on another side I was promoting people to use Cucumber BDD frameworks! A combination of both would be awesome! I am certainly not interested to use Tallyb's framework as it's becoming difficult to use it with the latest playwright features! But she certainly did a good job making one for people!

[PavanMudigondaTR]

@pavelfeldman @mxschmitt Please let us know if this will be taken up anytime soon?

[holdfenytolvaj]

Is there testing without BDD?

[lthurr]

Also experiencing #14539 (Cannot use .toMatchSnapshot() outside of playwright test runner)

[gzuppl]

I'm in the same boat as many of the earlier posters: We would migrate to playwright instantly if there was "native support" for BDD - as in an adapter for Cucumber resulting in all playwright features being usable in tandem with test cases written in Gherkin.

[PavanMudigondaTR]

I have ended up recommending to use WebdriverIO for one of the team who is currently using proctractor cucumber framework. It would have been awesome if playwright test can offer cucumber out of box. May be in future I can recommend some teams to use Playwright Test instead.

[SabFloki]

It would be cool if BDD with playwright comes handy. I have tried with cucumber - playwright part , yet I personally feel, we would miss the stunning use cases of playwright test runner. Please consider to plugin BDD with playwright.

[PavanMudigondaTR]

@pavelfeldman @mxschmitt Please let us know if this is on the roadmap. I have a team of 200 developers and testers who approached me for automation solutions so I was thinking about this.

[dsumac]

In the mean time, it exists an acceptable alternative: use the playwright steps to explain all Given/When/Then assertions.

The solution is light and fonctionnal.

Ex:

import { test, expect } from '@playwright/test';

test('Scenario explanation', async ({ page, context  }) => {

  await test.step('Given ...', async () => {
    ...
  });

  await test.step('When user make a stuff', async () => {
    ...
  });

  await test.step('Then a stuff is displayed on page', async () => {
    ...
  });

  await test.step('And user click on a button', async () => {
    ...
  });

  await test.step('Then an another stuff is displayed', async () => {
    ...
  });
});

And the report (with html reporter) is very readable.

[PavanMudigondaTR]

In the mean time, it exists an acceptable alternative: use the playwright steps to explain all Given/When/Then assertions.

The solution is light and fonctionnal.

Ex:

import { test, expect } from '@playwright/test';

test('Scenario explanation', async ({ page, context  }) => {

  await test.step('Given ...', async () => {

    ...

  });

  await test.step('When user make a stuff', async () => {

    ...

  });

  await test.step('Then a stuff is displayed on page', async () => {

    ...

  });

  await test.step('And user click on a button', async () => {

    ...

  });

  await test.step('Then an another stuff is displayed', async () => {

    ...

  });

});

And the report (with html reporter) is very readable.

Yes, I am using this currently. Currently this approach is causing me to repeat code and maintenance difficult. Just trying to see if cucumber support would remove repeating same code related to given when then. Any how thanks for inputs.

[TakesTheBiscuit]

Up

[tonirecio]

Up

[dsumac]

In the mean time, it exists an acceptable alternative: use the playwright steps to explain all Given/When/Then assertions. The solution is light and fonctionnal. Ex:

import { test, expect } from '@playwright/test';

test('Scenario explanation', async ({ page, context  }) => {

  await test.step('Given ...', async () => {

    ...

  });

  await test.step('When user make a stuff', async () => {

    ...

  });

  await test.step('Then a stuff is displayed on page', async () => {

    ...

  });

  await test.step('And user click on a button', async () => {

    ...

  });

  await test.step('Then an another stuff is displayed', async () => {

    ...

  });

});

And the report (with html reporter) is very readable.

Yes, I am using this currently. Currently this approach is causing me to repeat code and maintenance difficult. Just trying to see if cucumber support would remove repeating same code related to given when then. Any how thanks for inputs.

To avoid code repeating on "given/when/then" you can extract the steps like this:

async function checkStep(page: Page) {
  await test.step('When ... common step', async () => {
    await ...
  });
}

and use it in tests:

test('My test which is extracted steps', async ({ page }) => {
  await checkStep(page);
});

Plus, if you need, you can even implement the Page Object Pattern

[natalygoloborodko]

Using test.step() doesn't solve the problem. Many companies use protractor + cucumber and business wants to see those Gherkins as they were after switching to Playwright. It may work for completely new projects, but for migrating existing ones... no.

[PavanMudigondaTR]

Using test.step() doesn't solve the problem.

Many companies use protractor + cucumber and business wants to see those Gherkins as they were after switching to Playwright.

It may work for completely new projects, but for migrating existing ones... no.

Right now I need for two business units who are on proctractor and selenium with cucumber. So test.step is not solving it. I am trying WebdriverIO but it's not as good as playwright but I am not having choice other than going with WebdriverIO .

[mahula]

Up

[revathyvarma12]

is there any way to run cucumber test parallel with playwright

[benjefferies]

is there any way to run cucumber test parallel with playwright

Cucumber takes an argument --parallel 3 which you can use to run tests in parallel https://github.com/cucumber/cucumber-js/blob/main/docs/parallel.md

[oliverxu2008]

yes, bdd is good to have for playwright.

[mortelee]

The strong point of using cucumber is that you use feature files to explain what is tested. Using test.step and even extracting steps so that they can be re-used is a small step for software engineers to make the test better readable but is still not that what we find in cucumber and using feature files. Tally's example of how to use cucumber icw playwright works for me (https://github.com/Tallyb/cucumber-playwright) and I don't understand why so many people want to have native support from playwright/test

[lthurr]

The suggestion from @mortelee works fine. 👍

However, there are some things that we can't use outside of Playwright runner, like:

.toMatchSnapshot(name[, options])
.toMatchSnapshot([options])
.toHaveScreenshot(name[, options])
.toHaveScreenshot([options])

Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.

[benjefferies]

The suggestion from @mortelee works fine. 👍

However, there are some things that we can't use outside of Playwright runner, like:

.toMatchSnapshot(name[, options])
.toMatchSnapshot([options])
.toHaveScreenshot(name[, options])
.toHaveScreenshot([options])

Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.

To get around the snapshot stuff I created a function

function toMatchSnapshot(name: string, actual: any) {
  const file = `${process.cwd()}/snapshots/${name}.json`;
  if (!fs.existsSync(file)) {
    fs.writeFileSync(file, JSON.stringify(actual, null, 2));
  }
  const expected = require(file);
  expect(actual).toMatchObject(expected);
}

Then('the BEE configuration has rules', function (dataTable: DataTable) {
  const [{ id = 1 }] = dataTable?.hashes()?.length === 0 ? [{}] : dataTable.hashes();
  const name = id ? `${this.cucumberScenarioName}-${id}-rules` : `${this.cucumberScenarioName}-rules`;
  toMatchSnapshot(name, this.beeConfiguration.rules);
});

[yaffol]

I am using https://github.com/Tallyb/cucumber-playwright to integrate Playwright and Cucumber. The biggest thing for me that is missing is the integration with the VSCode extension - I liked being able to run/debug tests direct from the test explorer in VSCode, whereas now I must run and debug all my tests via the command line. This isn't a huge difficulty, but it's a shame, and it does make for more friction when working with tests. I'd like to be able to run my BDD/gherkin-based tests from within test explorer as well.

[mechandan]

Yes this feature is must , microsoft team please bring a native support playwright + BDD … thanks

[evertonlperes]

I agree with the folks, this feature will help a lot of people where BDD is a requirement.

Currently I'm in the same situation as many folks here, using cucumber-js with a customized world config (a bit overcomplicated due multiple browsers vs devices).

Hope some solution comes up soon. Excited to be a beta tester for it. I'm currently replacing from an old Java8+Selenium+Cucumber framework to Typescript + Playwright + Cucumber (around 5K test scenarios).

[JoaoEnesGoncalves]

Please, please! playwright with a native integration with cucumber/BDD/Gherkin it will be extremely helpful !

[NikkTod]

Hello great PW team, this has over 150 votes, probably the most voted thing , how much more are needed ;)

@mxschmitt @aslushnikov

[SpiderRam]

Adding another vote for this. Strongly second the comment about losing the functionality of the extensions. And it seems that the custom error messaging on expects does not come through, either. If I throw an error, it shows up as expected. But the custom expect message is lost completely.

[PavanMudigondaTR]

@mxschmitt @aslushnikov

Please let us know your opinion on this feature request ? I was wondering if this will happen in next 12 months !

[Tallyb]

I am using https://github.com/Tallyb/cucumber-playwright to integrate Playwright and Cucumber. The biggest thing for me that is missing is the integration with the VSCode extension - I liked being able to run/debug tests direct from the test explorer in VSCode, whereas now I must run and debug all my tests via the command line. This isn't a huge difficulty, but it's a shame, and it does make for more friction when working with tests. I'd like to be able to run my BDD/gherkin-based tests from within test explorer as well.

@yaffol My demo repo contains a way to run the tests directly from vscode, by using the debugger: https://github.com/Tallyb/cucumber-playwright#in-visual-studio-code (for the record: I also upvoted this feature, but until we have it, you are welcome to use my repo).

[csabasolyom]

The suggestion from @mortelee works fine. 👍 However, there are some things that we can't use outside of Playwright runner, like:

.toMatchSnapshot(name[, options])
.toMatchSnapshot([options])
.toHaveScreenshot(name[, options])
.toHaveScreenshot([options])

Visual comparison is a really nice functionality of playwright that we can't use if we happen to have a different runner on our projects.

To get around the snapshot stuff I created a function

function toMatchSnapshot(name: string, actual: any) {
  const file = `${process.cwd()}/snapshots/${name}.json`;
  if (!fs.existsSync(file)) {
    fs.writeFileSync(file, JSON.stringify(actual, null, 2));
  }
  const expected = require(file);
  expect(actual).toMatchObject(expected);
}

Then('the BEE configuration has rules', function (dataTable: DataTable) {
  const [{ id = 1 }] = dataTable?.hashes()?.length === 0 ? [{}] : dataTable.hashes();
  const name = id ? `${this.cucumberScenarioName}-${id}-rules` : `${this.cucumberScenarioName}-rules`;
  toMatchSnapshot(name, this.beeConfiguration.rules);
});

@benjefferies Did you use this for image (screenshot) comparison? Could you give some more context on this? e.g. what is in beeConfiguration.rules and why are you loading .json file? thanks

[benjefferies]

@csabasolyom I haven't use it for image comparison. All I've done is create a custom snapshot assertion check similar to https://jestjs.io/docs/snapshot-testing

[refactoreric]

Built-in support for BDD would be great. Pure frontend tests can do without, but for real end-to-end tests on more complex applications, you need your tests to speak the language of the domain, and you need more layers in your test automation architecture. Something like specs/feature files (tests), step definitions, page objects and API clients. It doesn't necessarily have to be a DSL like Cucumber though. But some guidance and support in the framework how to build such test automation architecture with Playwright would be welcome.

[josempd]

This would be great to have! I'm also using another implementation, but BDD is important for scalability.

[AbhishekPalani]

I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.

[Agustin-DeLuca]

I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.

I love Playwright but I really can't understand why the devs won't say anything about their most commented issue.

[GeGerardo]

Hi @pavelfeldman how are you doing? Do you have any update regarding this feature request?

[TimvdEijnden]

When comparing solutions for E2E testing I found that webdriverIO & cypress support cucumber through the packages: @badeball/cypress-cucumber-preprocessor @wdio/cucumber-framework

In playwright it's possible using cucumber-js but you'll miss all the advantages of the playwright test runner. So an implementation within playwright increase the change ofus migrating to it :)

[PavanMudigondaTR]

I keep coming back to this issue thread everyday hoping for a confirmation/ETA/resolution from the Playwright Team. A native support for BDD style playwright test runner will encourage many QAs to migrate existing selenium or Cypress based automation solutions.

I love Playwright but I really can't understand why the devs won't say anything about their most commented issue.

This could be complex and complicated to implement otherwise they have would done it already as its been a year now since this request was originally logged.

[Selenium-ToolBox]

I really love playwright, but sadly because of this feature not being implemented, I can't offer this as a Test Automation solution for my company.

[arvinder06]

Please add this feature.

[MartinB134]

The handling of the page object from the session is far from perfect and also adding new URLs to the pytest objects as property should be changed (don't know how to do it better), but this is the only option for now, if you want to have BDD, Data Driven testing, Playwright and Allure reporting. I know there is much work to do (dead code, IDE integration). But my resources are limited during my free time: https://github.com/MartinB134/playwright-pytest I will appreciate any Feedback and participation!

[3ackdoor]

Do you guys @pavelfeldman @aslushnikov have any update for this request? This has been requested a year for now and I see there are so many people upvoted for this so.

[avinashdj]

Any udpate on this. Eagerly waiting.

[VitaliyPotapov]

While waiting official comment I'd suggest to discuss, what actually do you expect here. I see several options:

1. Using playwright test-runner that reads and executes BDD tests written in gherkin
2. Using cucumber test-runner that have easy way to integrate playwright in steps

[mechandan]

1. Using

Yes we can do that as well. I would prefer playwright test runner if we can make it in anyways 😊

[NikkTod]

I think we are waiting for the first option -> 1. Using playwright test-runner that reads and executes BDD tests written in gherkin

[bluvulture]

Hello,

Ideally this would be also backwards compatible with current tests implementations, meaning steps or assertions could be marked as a BDD step.

Use case: We have already added cucumber-js within our test suite cause it was a business requirement, and we are using same test base with cucumber and playwright runner. Cucumber is used for business people, playwright for everything else. I would hate to need to rewrite everything that we have currently written to fit it in, would probably stick to current implementation in that case.

Example

Base function

export const clickAgreeWithDisclaimer = async ({ page }: TestArgs): Promise<void> => {
    await page.getByTestId(AGREE_BUTTON).click();
};

Cucumber feature file

  Background:
    Given I agree with disclaimer

BDD step

Given('I agree with disclaimer', () => agreeWithDisclaimer({ page, context }));

Playwright

test.describe('Feature: Overview', () => {
    test.beforeEach(async ({ page, context }) => {
        await agreeWithDisclaimer({ page, context });
    });
});

[yaffol]

If I was to describe my ideal developer experience in terms of using the tools, it would be to be able to use playwright's vs-code extension to see and run my tests, with the great tooling like tracing, but to be able to run tests derived from BDD features there.

And also to be able to run them via the CLI, but that goes without saying.

I guess what I am trying to say is that I really appreciate the DX (Testing eXperience?) of playwright/test, the trace viewer and now UI ('watch') mode, and would really like to be able to use them as well as tests that are executable specifications.

[NikkTod]

Hey guys @dgluzman, @pavelfeldman, @aslushnikov, have you ever discussed that feature in your team meetings, please please share some update on it, many people are wondering if there is a chance to see it.