Output failed command number for selenium-side-runner
Motivation
Other tools used for End User Monitoring usually output the failed step or command number. selenium-side-runner currently outputs the failure in a "code-like" format.
Making selenium-side-runner more friendly when performing EUM, an output that includes failed step/command number would be beneficial to identify which part of the SIDE project failed to complete successfully.
Current jest output is 75% of the way there, it just needs another field showing which numbered command (or description of command) failed.
Example
My example is separated out of the code block and in bold. The text is just an example, but at a minimum the failed command/step number, and additionally the command set (i.e verify text -> element -> value expected).
The audience of the extra output would be non-technical users to be able to interpret the failure in layman's terms with minimal requirement to understand the technical flow of the test.
$ selenium-side-runner -s http://localhost:4444/wd/hub myproject.side --output-directory test --output-format junit
info: Running myproject.side
added 26 packages from 24 contributors and audited 26 packages in 2.297s
found 0 vulnerabilities
FAIL ./DefaultSuite.test.js (14.377s)
Default Suite
β MyTest (11494ms)
β Default Suite βΊ MyTest
expect(received).toHaveText(expected)
Expected value to be (using Object.is):
"does not exist"
Received:
"this text is present"
31 | await driver.wait(until.elementLocated(By.xpath(`//div[@id=\'copyright-wrapper\']/div/div/div/div`)), 30000);
32 | await driver.wait(until.elementLocated(By.css(`.col-md-6 > div`)), configuration.timeout);
> 33 | await expect(driver.findElement(By.css(`.col-md-6 > div`))).resolves.toHaveText(`does not exist`);
| ^
34 | }
35 | module.exports = tests;
at Object.toHaveText (../../../usr/local/lib/node_modules/selenium-side-runner/node_modules/expect/build/index.js:202:20)
at Object.<anonymous>.tests.MyTest (commons.js:33:72)
Test Suites: 1 failed, 1 total
Tests: 1 failed, 1 total
Failed Step: 8 (verify text -> css=.col-md-6 > div; value="does not exist")
Snapshots: 0 total
Time: 14.598s, estimated 15s
Ran all test suites.
To counter myself, I could do something like grep the output for the > and then use that as the error report, however out of context (without knowing the rest of the script) a layman might not understand what it means.
π Feature Proposal
Output failed command number for selenium-side-runner
Motivation
Other tools used for End User Monitoring usually output the failed step or command number. selenium-side-runner currently outputs the failure in a "code-like" format.
Making selenium-side-runner more friendly when performing EUM, an output that includes failed step/command number would be beneficial to identify which part of the SIDE project failed to complete successfully.
Current jest output is 75% of the way there, it just needs another field showing which numbered command (or description of command) failed.
Example
My example is separated out of the code block and in bold. The text is just an example, but at a minimum the failed command/step number, and additionally the command set (i.e verify text -> element -> value expected).
The audience of the extra output would be non-technical users to be able to interpret the failure in layman's terms with minimal requirement to understand the technical flow of the test.
Failed Step: 8 (verify text -> css=.col-md-6 > div; value="does not exist")