Missing Test Items when using Async Reporting and Reruns

[MattMolloyatHilti]

We are running into an issue where test items don't show up in a launch when the launch is rerun and we are using async reporting. I have managed to debug the issue and found that it has to do with the RerunHandler modifying the uuid of the existing launch.

Here is the scenario:

1. First launch starts
2. Test items are started and finished
3. First launch finishes
4. Second launch starts right after, marked as a rerun
5. Failed test items from the first launch are retried
6. Second launch finishes

Since the first and second launch have different uuids, it is possible for their messages to end up in different queues. What can happen is that the queue used by the first launch can still have a bunch of messages related to that launch in it when the START_LAUNCH message for the second launch is processed by the RerunHandler, which then changes the uuid of the first launch. When messages related to the first launch then continue to be processed, they can no longer find their launch since the uuid has changed. This results in missing test items.

Workaround:

Currently we are working around this by limiting our report portal installation to a single queue, but this is very limiting and we would prefer to be able to use multiple queues to help with the volume of launches we are dealing with.

Possible solution:

Perhaps an alias table could be created that maps an old uuid to the replacement uuid. An entry could be inserted when a rerun launch is started and the original launch is not marked as finished. When the FINISH_LAUNCH message for the original launch's uuid is received, its entry in the alias table could then be removed.

[Pink-Bumblebee]

@MattMolloyatHilti , сurrently, this is a feature of asynchronous reporting. In situations like yours, we recommend using synchronous reporting. Unfortunately, at present, the proposed method using an alias table is not being considered for implementation.