Allow the user to specify a custom command to execute tests

[argentlynx]

In the future is there any plan to allow a custom command when executing tests? I have some tests that need to be run with cucumberjs and others require nightwatchjs as the command (which then calls cucumber).

Wasn't sure how to raise this as a feature request, sorry if it's the wrong place feel free to just delete this after you read it.

[Balrog994]

Hi @argentlynx

I have no experience using nightwatchjs, so I actually don't know if your request could be easly implemented. There are only 2 mandatory features required to run the tests:

• The command must support a parameter that filters the tests to be ran by title
• The command output must be compatible with the cucumberjs json message format

If this is achievable with nightwatchjs than I guess we can implement something like that. I was privately testing something like you requested to enable code coverage while running cucumberjs tests using a third party executable.

Can you please give some feedback about the requirements?

Thanks 👍

[argentlynx]

Thanks @Balrog994

I downloaded the sourcecode for cucumber-test-runner to try to implement it. I might have something that works but still experimenting with it.

Nightwatch is actually kind of a pain to do this with. It does not accept cucumber *.feature files as parameters, only tags. So I'm messing around with converting the filename and test line number into a tag to pass as a parameter to nightwatch (and then I'll just tag the scenarios in cucumber with the filename)

Not sure if it's a good fit really, so I don't know how much time I'll dedicate to this, but here is what I did so far in case anyone gets interested in this or if you're interested in looking into this later if people want it.

testRunner.ts

1. @line:187 itemsOptions - convert this to tags const itemsTags = itemsOptions.map(x => "@" + x.split("\").pop().replace(/[.: -]/g,'_'));

2. then itemsTags as a parameter for nightwatch

var tagParameter = "--tags " + itemsTags[0];

(or you could use itemsTags.join(" AND") but I think there is a bug in nightwatch not allowing multiple tags to be passed for the moment so I'm just passing a single tag.

3. @line:211 Modify the cucumberProcess = spawn( to...

   const cucumberProcess = spawn(
       `npx`,   // we need to run nightwatch with npx, not with node
       [[], `${workspace.uri.fsPath}/node_modules/nightwatch/bin/runner.js`, tagParameter, "", "", []],
       {
           cwd: workspace.uri.fsPath,
           env: env,
       }
   );

4. [haven't tried yet] (possibly) detect if a tag is matching the feature, if so, spawn with npx nightwach, if not, spawn normal node cucumber

References

Installing the nightwatchjs project with cucumberjs support https://www.browserstack.com/guide/nightwatchjs-tutorial

• after installing this project, I can see the nightwatch tests in the cucumber-test-runner - but they can only be run via commandline npx nightwatch --tags @something

So maybe the best thing for the time being is I will just hide the nightwatch tests in the test runner. Not sure yet. Leaving my notes here in case someone else is trying it.

[argentlynx]

OK got a draft working but using the minified code. It's a one-liner but if you want to somehow make a feature for this in the future I donate it to the code penny jar.

after installing the nightwatch project (at url listed above)

1. Edit your nightwatch.feature file, add tag @nightwatch_feature_8 to the scenario "Searching the Rijksmuseum" (because Scenario: Searching the Rijksmuseum appears on the .feature file line 8)

2. goto the .vscode/extensions/balrog994.cucumber-test-runner-0.5.0/dist/main.js file

3. remove this line

let w=(0,Va.spawn)("node",[...R,${a.uri.fsPath}/node_modules/@cucumber/cucumber/bin/cucumber.js,...n,"--format","message",...y],{cwd:a.uri.fsPath,env:c});

4. replace it with

let w=(0,Va.exec)("npx " + ${a.uri.fsPath}/node_modules/nightwatch/bin/runner.js + (" --tags \"" + n.map(x => "@" + x.split("\").pop().replace(/[.: -]/g,'_'))[0] + "\""),{shell:true,cwd:a.uri.fsPath});

5. Save

6. Re-open VS Code

7. Open cucumber-test-runner

8. Click "Run" icon next to the nightwatch.feature - Google Search test

Done, now it works (but the code is rough)

C:\WINDOWS\system32\cmd.exe C:\WINDOWS\system32\cmd.exe,/d,/s,/c,"npx c:\Users{youruseridinwindows}{pathtoyournightwatchfolder}\NightwatchTestFolder/node_modules/nightwatch/bin/runner.js --tags "@nightwatch_feature_8"" stdout: Using: chrome (114.0.5735.199) on WINDOWS. stdout: ...√ Testing if the page title equals 'Rijksmuseum Amsterdam, home of the Dutch masters' (56ms) stdout: .√ Element <#rijksmuseum-app> was visible after 33 milliseconds. stdout: .√ Testing if element <.search-results> contains text 'Operation Night Watch' (1263ms) stdout: .. stdout: 1 scenario (1 passed) stdout: 5 steps (5 passed) stdout: 0m08.358s (executing steps: 0m08.341s) stdout:  Wrote HTML report file to: c:\Users{whateveryourpathistothisfolder}\NightwatchTestFolder\tests_output\nightwatch-html-report\index.html stdout: 

If this becomes a feature I imagine it would have to have a setting or something to enable it in the settings page, and be able to detect whether or not the filename matches a tag to decide whether to run it using "npx nightwatch" vs. "node cucumber" or something like that, dunno but there it is. : D