[Feature] repeatEach / --repeat-each should respect test.describe.serial and run the repeats serially

[mstepin-edt]

Currently if you define a test or group of tests with test.describe.serial or test.describe.serial.only, and then run the suite with repeatEach or CL --repeat-each, the repeats are run in parallel to each other.

I would expect in this case for the repeats to continue to respect the serial configuration for the test or group of tests.

[dgozman]

@mstepin-edt When I run this scenario locally, serial tests are always run in the correct order. However, each serial scenario is repeated with --repeat-each, and Playwright utilizes available workers to run these series in parallel.

If you'd like to run all the tests serially, you can pass --workers=1 CLI flag. Let me know if that helps.

[mstepin-edt]

Not really going to solve the problem as then all tests will be run in series. So if I have 10 spec files, and one of them is set up with test.describe.serial while the others are all parallel, I would still expect that to be respected with the --repeat-each flag. I was very close to logging this as a bug rather than a feature request. Having the tests run 10 times slower isn't really an option here.

[dgozman]

@mstepin-edt Thanks for the explanation. I'll leave this issue open for prioritization.

[DaBittna]

A (sort of) workaround I found: Use test.skip() in combination with repeatEachIndex. By checking for a specific worker index, you can make the test run only once, it will be skipped on the other workers.

Now, you still can't really use --repeat-each for that test (the feature request still stands), but at least you can use it for the rest of your test suite without breaking.

test.beforeEach(async ({page}, testInfo) => {
  test.skip(testInfo.repeatEachIndex > 0, 'cannot be parallelized');

  await page.goto('/');
  // ...other setup
});

test('my test', async () => {
  // test
});

[seahindeniz]

Is this issue going to be prioritized? Most of my test cases are serial and running tests multiple times results in each case running multiple times. Instead, it should have run the .serial block N times, not the each test