Add commands to create a new test run

[jakec-routable]

What would you like the TestRail CLI to be able to do?

I would like the ability to use the trcli to create a new test run before using the command for uploading results. The output of the command would be the run id. Command arguments would match the API request body parameters for add_run.

Example command usage:

$ trcli -y \
>  -h https://INSERT-INSTANCE-NAME.testrail.io \
>  --project "My Project" \
>  --username INSERT-EMAIL \
>  --password INSERT-PASSWORD \
> add_run \
> --name "Test run name" \
> --include_all \
> --refs "SAN-1, SAN-2"

Why is this feature necessary on the TestRail CLI?

This step will be used in CI/CD pipelines prior to uploading test results, so that test result uploading can be performed across a number of CI/CD nodes that execute in parallel and all nodes can report into the same test run using the created id.

For example, if I split a set of pytests across 3 nodes, each node will generate their own results.xml file. Using this command, I will be able to upload all results into the same test run following these steps:

1. Use trcli create_run command to create the test run.
2. Pass the test run id to each test node.
3. Tests execute across test nodes.
4. When each test node finishes, it reports results from its own generated results.xml file into the test run created in the first step.

More details

No response

Interested in implementing it yourself?

Yes

[jakec-routable]

The original issue description included test plans and test runs, but I have narrowed the scope of this issue to only test runs.

[Testinator-X]

If I understand the ticket here correctly, your use case seems to be going in the same direction as the use case I described here:

https://github.com/gurock/trcli/issues/149#issuecomment-1781810531

However, the problem with your request may be that the TestRail application itself does not allow you to create empty runs without tests. To be honest, I'm not sure whether this is possible via the API. However, I am relatively sure that it is not possible via the web interface and that you have to add at least one test to the run.

As a workaround, you would first have to create a temporary run for each results.xml file, then create your own run with all TestCases from the temporary runs via the TestRail API without the CLI tool, then bundle your results in the desired common run and finally delete the temporary runs again via the API.

It would also be very helpful if it were finally possible to create dynamic TestRuns via the TestRail API and that new cases that match the filter criteria of the run are finally added to the run automatically.

[jakec-routable]

Test runs can be created via the API without test cases. The UI requires a selection of either specific test cases or including all test cases, but the API does not require either. A run can be created without any test cases added to it. Then updated with the addition of test cases.