gurock / trcli

TR CLI (trcli) is a command line tool for interacting with TestRail.
Mozilla Public License 2.0
47 stars 42 forks source link

Testcases with same name but different automation_id are duplicated #103

Closed pixilation closed 1 year ago

pixilation commented 1 year ago

TestRail CLI Version

1.3.0

CLI Environment

Windows 11, Python 3.10.4

TestRail Version

7.5.5

TestRail Instance Type

Professional Cloud

Current behavior

Testcases with duplicate names (but unique name+classname / automation_id) are causing duplicate testcases to be added. This behaviour occurs when a new testcase is added. When there are no new testcases to add it doesn't happen.

When this happens the CLI shows this: "Warning: Case duplicates detected in mytests.xml. This will result in improper results setting."

Any occurrences of a testcase name beyond the first are duplicated.

The run will only include results for 1 of each name, regardless of how many occurrences there are.

<testsuites>
    <testsuite name="NewTest">
        <!-- Increment this to trigger bug -->
        <testcase name="Incrementing Test Number 01" classname="NewTest"/>
    </testsuite>
    <testsuite name="Professional">
        <testcase name="Test 1" classname="Professional"/>
        <testcase name="Test 2" classname="Professional"/>
        <testcase name="Test 3" classname="Professional"/>
    </testsuite>
    <testsuite name="Enterprise">
        <!-- These are duplicated everytime 1 or more new tests are added -->
        <testcase name="Test 1" classname="Enterprise"/>
        <testcase name="Test 2" classname="Enterprise"/>
        <testcase name="Test 3" classname="Enterprise"/>
    </testsuite>
    <testsuite name="Basic">
        <!-- These are duplicated everytime 1 or more new tests are added -->
        <testcase name="Test 1" classname="Basic"/>
        <testcase name="Test 2" classname="Basic"/>
        <testcase name="Test 3" classname="Basic"/>
    </testsuite>
</testsuites>

Desired behavior

I would expect that no duplicates are created, and that the run includes all the results.

More Details

No response

d-rede commented 1 year ago

Hi there @pixilation

Thanks for submitting this! Version 1.3.1 should also have this one fixed, do you mind confirming and closing the issue if everything's ok? :)

pixilation commented 1 year ago

Thanks, I can confirm this is working as expected now.

The message "Warning: Case duplicates detected in collated_tests.xml. This will result in improper results setting." is appearing when the above steps are run, but there are no duplicates added. I think this can probably be removed, at least for this scenario. I'm not worried about it though.