run the Playwright lib-ui and app tests in their own job, parallel to the frontend and backend jobs
run the Playwright e2e tests across three shards
About sharding
Because we tests against three browsers and use three shards, each shard now covers one browser. We could not use shards (yet) and create a separate job for each browser. At some later point we could then shard each of the browser jobs. I didn't see an advantage to doing it that way. And the current way makes it really easy to add a shard, i.e. add a number to shard: [1, 2, 3].
About caching
I tried installing and chaching node modules and browser engines in a job. There's several issues with this:
The app tests fail.
Webkit caching does not work, because Playwright installs dependencies outside of the cached folder ~/.cache/ms-playwright. I didn't try to find out how many additional folders would need to be cached to get it to work.
The difference between installing Webkit and installing all three browser engines (Chrome, Firefox, Webkit) is quite small. So caching node modules / browser engines and the installing Webkit on every runner that needs it, does not result in a significant speed increase.
So caching node modules and browser engines is not worth it.
Scope:
About sharding Because we tests against three browsers and use three shards, each shard now covers one browser. We could not use shards (yet) and create a separate job for each browser. At some later point we could then shard each of the browser jobs. I didn't see an advantage to doing it that way. And the current way makes it really easy to add a shard, i.e. add a number to
shard: [1, 2, 3]
.About caching I tried installing and chaching node modules and browser engines in a job. There's several issues with this:
~/.cache/ms-playwright
. I didn't try to find out how many additional folders would need to be cached to get it to work.So caching node modules and browser engines is not worth it.
Main branch:
This branch: