[Bug]: Playwright not sharding tests evenly

[rohitkrishna094]

Version

^1.48.0(latest)

Steps to reproduce

Example steps to reproduce.

1. Copy this code into a file called tests/shard-test.spec.js

   
   const { test, expect } = require('@playwright/test');

const arr = Array.from({ length: 2000 }, (_, i) => i + 1);

test.beforeAll(async () => { console.log('Before tests'); });

arr.forEach((num) => { test(@smoke should check Math.pow functionality for ${num}, async ({ page }) => { expect(num ** 2).toBe(Math.pow(num, 2)); }); })

2. Then run the test using this command `npx playwright test tests/shard-test.spec.js --shard=2/8`
3. Look at the logs and you will notice that no tests run.
4. However on shard 1 it gets 500 tests allocated to it(when in fact each shard should get 2000/8 = 250 tests)

### Expected behavior

- Each shard should get same amount of tests allocated to it.
- So in this example each shard should get 2000/8 = 250 tests each.
- And finally some shards should not exit immediately `doing no work`

### Actual behavior

- Bug in playwright where tests are not sharded evenly
- Noticed that this happens if I added test.beforeAll in this file. If I comment that out, then it works fine.

### Additional context

_No response_

### Environment

## System:
 - OS: Windows 10 10.0.19045
 - CPU: (8) x64 Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz
 - Memory: 6.76 GB / 15.88 GB
## Binaries:
 - Node: 20.16.0 - C:\Program Files\nodejs\node.EXE
 - npm: 10.8.2 - C:\Program Files\nodejs\npm.CMD
## IDEs:
 - VSCode: 1.94.0 - C:\Users\Owner\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
## Languages:
 - Bash: 5.1.16 - C:\WINDOWS\system32\bash.EXE
## npmPackages:
 - @playwright/test: ^1.48.0 => 1.48.0

[rohitkrishna094]

This seems like an issue from few versions ago since the new algorithm is to shard files as opposed to tests https://github.com/microsoft/playwright/issues/21497 Comment

But even that seems to not be the correct statement since if it was true, then in my example the 1st shard should get all the tests while the remaining shards should get 0 tests. However seems like there is some form of hybrid shading algorithm at play here.

Besides that, there has to be a way to shard tests evenly even if I am using a single file because there are many usecases where users might generate tests dynamically and still have beforeEach or beforeAll in their test files.