gurock / trcli

TR CLI (trcli) is a command line tool for interacting with TestRail.
Mozilla Public License 2.0
48 stars 39 forks source link

With code first approach, trcli creates duplicates of test cases then chokes on deadlock trying to cleanup and fails #233

Open latte-incognito opened 1 month ago

latte-incognito commented 1 month ago

TestRail CLI Version

1.9.5

CLI Environment

Mac OS Sonoma 14.3, Python 3.12.1

TestRail Version

6.7.2.1037

TestRail Instance Type

Professional Cloud

Current behavior

While uploading test results - 17 (test cases), with code first approach (test cases will be created by trcli)

trcli creates duplicates, 3 for each of them, then fails in the midst of the process, with 500 error in logs

Desired behavior

Desired behaviour test case duplication impossible via trcli.

More Details

My assumption is:

TRCLI send requests inside for loop (for retries purposes I assume) https://github.com/gurock/trcli/blob/d61901390dcd7b659f0dd6667e094263c53fb40f/trcli/api/api_client.py#L95

Particular add_case method inside creates ThreadExecutor with 10 workers https://github.com/gurock/trcli/blob/d61901390dcd7b659f0dd6667e094263c53fb40f/trcli/api/api_request_handler.py#L345

If one of those worker fails for any reason, full batch re-kicked again, with new iteration of the for loop without waiting for previous to finish.

Then Trcli tries to do the cleanup after failure, but can't do it because test rail instance returns 500 error (probably deadlock)

Also the possible reason of requests to failure in the first place can be that the Cloud instance of test rail have limits around of requests per minute ~150 smthng, with running 10 threads in for cycle can probably hit that limit very fast

P.s. I can't share neither logs, nor files, with NDA reason