search

gurock / trcli

TR CLI (trcli) is a command line tool for interacting with TestRail.
Mozilla Public License 2.0
47 stars 42 forks source link

Add option to disable deletion of test suite/case when upload fails #99

Open ian-ann opened 1 year ago

ian-ann commented 1 year ago

What would you like the TestRail CLI to be able to do?

I'd like to have an option not to delete create test suites when test run results failed to be uploaded

Why is this feature necessary on the TestRail CLI?

There are times that the account used is an app account with add/delete/edit role. At the same time, test suites are already available and being maintained. With just 1 error during test run upload, all the test suites that already been created will be deleted together with the test cases, section and test runs which is not a good idea specially if we are running 100+ tests. There is no need to delete the test suite just the test run should be enough for this case.

More details

Below is a sample of failed upload since the initial upload for the run-id has more than test cases/results than previous run.

`Checking project. Done. Adding missing sections to the suite. Found test cases not matching any TestRail case (count: 27) Adding missing test cases to the suite. Adding test cases: 27/27, Done. Adding results: 0/76 Error during add_results. Trying to cancel scheduled tasks.

Aborting: add_results. Trying to cancel scheduled tasks. Adding results: 0/76 Field :results contains one or more invalid results (case C497 unknown or not part of the test run) Deleted created test cases Deleted created section Deleted created suite`

Interested in implementing it yourself?

Maybe, let's talk!

Booklynn commented 1 year ago

@ian-an-github my issue is similar to yours, I just want to skip cases unknown, when adding test result. Fortunately, I solved my problem. https://github.com/gurock/trcli/pull/147

I would prefer to see this request merged into this repo. However, if it is not merged, I will continue using my custom library to meet the desired functionality.

Free925 commented 12 months ago

is there any news on this issue? Am I the only one that sees it as disruptive, since I see it labeled as "priority-low"? With the actual behaviour, if test automator fails to label correct testname of its automated test to update testrun, an entire test suite will be permanently deleted from the repository.

Mauales commented 12 months ago

Same here, this needs to be adressed to avoid the lost of critical data...high risk feature

bitcoder commented 9 months ago

Hi everyone, with v1.6.2, there was a bug fix for not deleting a suite unless it was created during the upload process. This will avoid the problem of deleting by mistake test suites that already existed before the upload process. Doesn't this address your concerns?