Preserve previously linked defects when updating an already existing Test Execution

[Aradesk]

Hi, Thank you for this plugin which is quite usefull. I'm opening an issue here since i don't know where else to put this one.

Context: In our work environment, we need to provide end-to-end automated tests (we use robot framework for that). Our Users wants to be able to create/prepare their Test Executions and then call the Jenkins API, to get their automated tests done and reported on the Test Execution key they specified.

That way, if they need to re-execute 1 or 2 automated tests in the list, they can do so easily and don't have to create a new Test Execution containing only those 1-2 tests.

Our issue:

When importing results to a specified Test Execution Key, if the Test in that Test Execution has already a linked defect from a previous execution, that linked defect is deleted.

How to reproduce:

Create a Test Execution "JIRA-XXXX" containing N Test in "TO DO" status

Execute the Jenkins Job that import results with Test Execution Key = "JIRA-XXXX" and use a Robot XML format.

• Test A has failed

Create/add a defect in the Execution Details of Test A. Change the status of Test A to "TO DO" and Execute the Jenkins Job again.

• Test A passed but the linked defect isn't there anymore

Asked Feature:

The X-Ray Import Result step shoud have a "Preserve Linked Issues" option (checkbox). When that option is checked and the Test Execution already exist, the plugin will :

• check if the test already exist in the Test Execution
• If the test already exist, it i will store the list of linked issues if that list is not empty
• It will then use that stored list when it calls the API to update the Test Execution

[Russell616]

Hi,

Thank you for your comment and for using Xray :)

In Xray, the "Test Run" page was created to represent a single execution, in a point in time. So Xray, by design, gives you 2 choices:

1. Always reuse the same Test Execution issue (i.e. always reuse the same Test Run) - like you are doing in your scenario. As I said before, a Test Run represents a Test Run in a given point in time, so, when Xray detects a new execution for the same Tests and Test Execution it will do a complete reset to do Test Run, removing all previous data (including previous Defects).
2. If you need to keep the history for the Test Run, including Defects, you would need to create a new Test Execution, this will keep the "execution history" that you can revisit at any time.

In you scenario if don't want to create a new Test Execution and you want to keep the Defects, I can make you a few suggestions:

• Link the Defects to the Test and/or Test Execution issue. The Test Run would still contain the information of the most recent execution and let the Test/Test Execution have the history of Defects
• If you don't need "formal" Jira Links, you can write in the Test/Test Execution an internal (private) comment with the defects found

We are always available to help you using Xray. If you want to discuss this further, I suggest you to open a support ticket with us here: https://jira.xpand-it.com/servicedesk/customer/portal/2

In our support system, we can talk more about your specific needs and find a more personalized solution to your situation.

[Aradesk]

Thanks for the reply. We thought about having the users linking issues to the Test Execution itself but have no way to enforce it ourselves. I guess we'll have to sort out the Best Pratice to use if we want KPIs that actually mean something.