Add cucumber option to cli-plugin-e2e-nightwatch

[brunogirin]

What problem does this feature solve?

This feature makes it possible to write e2e tests using gherkin feature files in addition to JavaScript.

Using gherkin feature files makes it possible to define e2e tests collaboratively with the non-technical members of a delivery team, in particular product owners and project managers, and contributes to better understanding of final functionality within the team.

What does the proposed API look like?

Addition of a boolean configuration option to cli-plugin-e2e-nightwatch (e.g. enable-cucumber). When that option is set, the following two additional packages are installed:

• nightwatch-cucumber
• cucumber

A slightly different NightWatch configuration file is used:

const nightwatchCucumber = require('nightwatch-cucumber');

nightwatchCucumber({
  cucumberArgs: [
    '--require', 'tests/e2e/features/step_definitions',
    '--format', 'json:tests/e2e/reports/cucumber.json',
    'tests/e2e/features',
  ],
});

module.exports = {
  src_folders: [],
  // remainder of configuration identical to the current configuration
}

A basic feature file is provided in tests/e2e/features:

Feature: Home page

  Scenario: Check expected elements on home page
    Given I open the home page
    Then I see the element ".hello"
    And the element "h1" contains text "Welcome to Your Vue.js App"
    And I count 1 element "img"

And corresponding steps definitions in tests/e2e/features/step_definitions (note that NightWatch custom assertion can still be used):

// Where the home steps will go
const { client } = require('nightwatch-cucumber');
const { Given, When, Then } = require('cucumber');

Given('I open the home page', function () {
    return client
        .url(process.env.VUE_DEV_SERVER_URL)
        .waitForElementVisible('#app', 5000);
});

Then('I see the element {string}', function (element) {
    return client.assert.elementPresent(element);
});

Then('the element {string} contains text {string}', function (element, text) {
    return client.assert.containsText(element, text);
});

Then('I count {int} element {string}', function (count, element) {
    return client.assert.elementCount(element, count);
});

The folder tests/e2e/specs is removed.

E2e tests can then be run as normal:

$ yarn run test:e2e
yarn run v1.5.1
$ vue-cli-service test:e2e --config nightwatch.config.js
 INFO  Starting development server...

 DONE  Compiled successfully in 2892ms                                                 15:52:50

  App running at:
  - Local:   http://localhost:8080/ 
  - Network: http://192.168.0.4:8080/

  App is served in production mode.
  Note this is for preview or E2E testing only.

Starting selenium server... started - PID:  3396

 ✔ Element <#app> was visible after 46 milliseconds.
.
 ✔ Testing if element <.hello> is present.
.
 ✔ Testing if element <h1> contains text: "Welcome to Your Vue.js App".
.
 ✔ Testing if element <img> has count: 1
.

1 scenario (1 passed)
4 steps (4 passed)
0m01.989s
Done in 8.97s.

[LinusBorg]

We're hesitant to add additional setups /options like this one to the cli base. Especially for a solution that (at least as far as my personal experience goes) isn't really widely used (a judgement stemming from the fact that I heard/read about this solution once, somewhere, but it's not a topic i come across regularly).

You could just as well write a plugin that augments the nightwatch configthat we set up.

[brunogirin]

@LinusBorg totally understand that and I would be happy to contribute a plugin. However, I'm very new to Vue and I'm not sure how I would go about creating a plugin that extends the e2e-nightwatch one. Would it be a case of creating a CLI plugin that has a dependency on @vue/cli-plugin-e2e-nightwatch and delegates everything to it apart from the configuration?

[Lorti]

I have started on a plugin for Cucumber, but it's still in its infancy, as I have encountered multiple problems. https://github.com/karriereat/vue-cli-plugin-e2e-nightwatch-cucumber

I plan on using nightwatch-api, the successor to nightwatch-cucumber, which is deprecated. Nightwatch API and the underlying Nightwatch 1.x don't have stable releases yet, so it will take some time before I can make the plugin usable, even if I find the time to work on it.

[brunogirin]

@Lorti that looks great, I'll check it out and see if I can make it work with my use case. I saw the shift from nightwatch-cucumber to nightwatch-api, hopefully Nightwatch 1.x will be stable soon and everybody can move forward.

@LinusBorg should I close this issue?

[Lorti]

@brunogirin I got the basic setup to work (with pre-defined tests) and will now start looking at generators, so one can write their own tests. After that I’ll have to add options and customization. If you want to chime in, be my guest ;)

[Lorti]

vue-cli-plugin-e2e-nightwatch-cucumber now has support for Nightwatch and Cucumber configuration and uses the latest Cucumber (5.0), Nightwatch (1.0) and Nightwatch API (1.0). Any feedback is greatly appreciated ;-)

[LinusBorg]

Closing as a community solution exists.