selenium-side-driver 4.0.0-alpha.15 with '--output-format' option fails with error 'unknown option'

[balimoon777]

🐛 Bug Report

selenium-side-driver 4.0.0-alpha.15 with '--output-format' option fails with error 'unknown option'

To Reproduce

Steps to reproduce the behavior: % selenium-side-runner --filter="happy" --output-directory="side_output" --output-format="junit" demo_project.side error: unknown option '--output-format=junit'

Note that the usage info of v4 doesn't contain the usage of '--output-format' while v3 does: BTW, typo in 'maximimum' in the description of '-t, --timeout [number]'

Usage: selenium-side-runner [options] your-project-glob-here-.side [variadic-project-globs-.side]

Options: -V, --version output the version number --base-url [url] Override the base URL that was set in the IDE -c, --capabilities [list] Webdriver capabilities -s, --server [url] Webdriver remote server -p, --params [list] General parameters -f, --filter [string] Run suites matching name, takes a regex without slashes, eg (^(hello|goodbye).*$) -w, --max-workers [number] Maximum amount of workers that will run yourtests, defaults to number of cores (default: 8) -t, --timeout [number] The maximimum amount of time, in milliseconds, to spend attempting to locate an element. (default: 15000) -x, --proxy-type [type] Type of proxy to use (one of: direct, manual, pac, socks, system) -y, --proxy-options [list] Proxy options to pass, for use with manual, pac and socks proxies -n, --config-file [filepath] Use specified YAML file for configuration. (default: .side.yml) -o, --output-directory [directory] Write test results to files, format is defined by --output-format -f, --force Forcibly run the project, regardless of project's version -d, --debug Print debug logs -h, --help display help for command

And below is the usage info in v3:

Usage: selenium-side-runner [options] project.side [project.side] [*.side]

Options: -V, --version output the version number -c, --capabilities [list] Webdriver capabilities -s, --server [url] Webdriver remote server -p, --params [list] General parameters -f, --filter [string] Run suites matching name -w, --max-workers [number] Maximum amount of workers that will run your tests, defaults to number of cores --base-url [url] Override the base URL that was set in the IDE --timeout [number | undefined] The maximimum amount of time, in milliseconds, to spend attempting to locate an element. (default: 15000) --proxy-type [type] Type of proxy to use (one of: direct, manual, pac, socks, system) --proxy-options [list] Proxy options to pass, for use with manual, pac and socks proxies --config, --config-file [filepath] Use specified YAML file for configuration. (default: .side.yml) --output-directory [directory] Write test results to files, format is defined by --output-format --output-format [jest | junit] Format for the output. (default: jest) --force Forcibly run the project, regardless of project's version --debug Print debug logs -h, --help output usage information

Expected behavior

selenium-side-runner command with '--output-format' option should be successful.

Project file reproducing this issue (highly encouraged)

Any valid .side file can be used to repro this issue.

Environment

OS: MacOS Monterey 12.6 Selenium IDE Version: v3.17.0 Selenium SIDE Runner Version: 4.0.0-alpha.15 Node version: v18.10.0 ChromeDriver version: 106.0.5249.61 Browser: Browser Version: 106.0.5249.61

[jezahl]

+1 for this issue

[danielapochini]

+1 this param is described in the official docs below and it leads the user to an error that it would be already available to be used https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner#output-test-results-to-a-file

[hxc-gxc]

+1 Same here cannot get any output report "output-directory=results" "--output-format=junit" >> option unknown

[toddtarsi]

This feature has little or no interest to me, but PRs are welcome.

[jezahl]

@toddtarsi OK, but how do you get any test output results then?

Doing this doesn't give anything in the results folder:-

selenium-side-runner -o results Simple.side

[toddtarsi]

I just use -d to get any debug info if I need it, and run tests primarily via a CI system, where the most important thing is exit codes and failure points. If there was anything I'd really want to improve, it would be the stack traces on failure.

[toddtarsi]

@jezahl - If you're really looking to do stuff, I left an escape hatch to pass in whatever jest CLI options you need (I use --forceExit personally because the browser vendor tunnels are a nightmare from jest and Github actions):

selenium-side-runner --jest-options '"--outputFile=./your-file.txt"' ./your-test.side

Shorthand version:

selenium-side-runner -j '"--outputFile=./your-file.txt"' ./your-test.side

[Baltazardoung]

@balimoon777 you can (with selenium-side-runner@4.0.0-alpha.23 or later) generate a junit report after installing jest-junit (npm install -g jest-junit) and configuring it.

JEST_JUNIT_OUTPUT_DIR=side_output JEST_JUNIT_OUTPUT_NAME=selenium-side-report.xml selenium-side-runner -j " --reporters=jest-junit  --reporters=default "  --filter="happy"  demo_project.side

[toddtarsi]

@Baltazardoung - You, sir or madam, are the real mvp. I can't praise you highly enough.

[toddtarsi]

I want to close this, but honest to god I don't want to lose your note @Baltazardoung

[jezahl]

@Baltazardoung This is great! Thanks

[chrisisth]

@balimoon777 you can (with selenium-side-runner@4.0.0-alpha.23 or later) generate a junit report after installing jest-junit (npm install -g jest-junit) and configuring it.

JEST_JUNIT_OUTPUT_DIR=side_output JEST_JUNIT_OUTPUT_NAME=selenium-side-report.xml selenium-side-runner -j " --reporters=jest-junit  --reporters=default "  --filter="happy"  demo_project.side

How do i do this ? "JEST_JUNIT_OUTPUT_NAME" is that a path variable ? I'm on Windows

[toddtarsi]

@chrisisth - That is an environment variable (https://docs.oracle.com/en/database/oracle/machine-learning/oml4r/1.5.1/oread/creating-and-modifying-environment-variables-on-windows.html)

[miiihi]

How about removing the option from the docs if there is no plan to implement it?

[toddtarsi]

@miiihi - I completely agree. Would you point me to the docs where it's at?

[miiihi]

Here https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner#output-test-results-to-a-file