[BUG]: Test retries are shown as multiple test cases

[cthlo]

Describe the bug Retried tests are rendered as separate test cases on the reporting dashboard. (Maybe it's a feature request or it's already in the backlog) Thanks!

To Reproduce Steps to reproduce the behavior:

1. Configure retries
2. Add a failing test

Expected behavior Retries should be grouped into the same test case.

Screenshots If applicable, add screenshots to help explain your problem.

Setup information (please complete the following information):

• Azure Subscription Id: N/A
• Workspace Name: N/A
• Service trace Id (if available):
• Log snippet (if available):
• Browser on which tests are executed (if applicable):
• Time range (in PST) when the issue was experienced:
• Playwright Version:
  • Client machine OS:
  • OS requested on Service:

Additional context Add any other context about the problem here.

[vvs11]

Hi @cthlo I can see the retries clubbed in the same test. Could you share the screenshot of how you are seeing it. A small repro for the issue along with your config would also help

[cthlo]

@vvs11 It seems that they're separated for my test retries when I have different (conditional) annotations added to the retries. Here's a simplified test:

import { test, expect } from "@playwright/test";

test.setTimeout(5_000);

test("timeout", async ({}, testInfo) => {
  if (testInfo.retry == 0) {
    testInfo.annotations.push({ type: "first run", description: "timeout" }); // annotation added to only first run
    await new Promise((resolve) => setTimeout(resolve, 10_000));
  }
  await expect(1).toBe(1);
});

[abranj19]

Thanks for reporting this bug. We will look into this and come back with an ETA for resolution.

[abranj19]

Tentative date for fix is ~15th November