search

gurock / trcli

TR CLI (trcli) is a command line tool for interacting with TestRail.
Mozilla Public License 2.0
47 stars 42 forks source link

Support for Custom Success and Failure Statuses #107

Open jhaffey2 opened 1 year ago

jhaffey2 commented 1 year ago

What would you like the TestRail CLI to be able to do?

TestRail allows for creating and using custom result statuses (see documentation). The TestRail CLI should be able to support these custom result statuses when reporting successes and failures.

Example: A TestRail Suite tracks test case results with 4 custom statuses: Manual Passed, Manual Failed, Automation Passed, Automation Failed.

Desired Result: After running tests with Cypress and generating results, the statuses for Passed should map to Automation Pass and for Failed it should map to Automation Failed.

Actual Result:: After running tests with Cypress and generating results, the results are uploaded as the default Passed and Failed statuses. Results must be updated manually if we want them to map to the desired statuses.

Why is this feature necessary on the TestRail CLI?

The CLI cannot be used efficiently for projects making use of custom test result statuses without a feature that supports it.

More details

Simple implementation suggestion: the CLI allows the user to specify an alternative status id for passed and failed.

Interested in implementing it yourself?

Maybe, let's talk!

ch4h2s commented 1 year ago

Also, for some types of tests, a custom comment may be needed, such as average time per request, average RAM used, total RAM, total time ....