Fix Playwright Test Suite

[jasonLaster]

The playwright test suite stopped running recently we should bring it back and ensure that we have telemetry wired up so we can be alerted in honeycomb if it stops running again

• Here are the docs on running the suite
• The suite should run on a variety of platforms, mac, linux, etc... but we can pick one for now (probably macstadium) and get that working again
• Backend runner (https://github.com/RecordReplay/backend/tree/master/test/runlive)

We have one telemetry event for when the live tests fail to create a recording

• https://github.com/RecordReplay/backend/blob/16a796be1a4d25f598cd17c2a77639e6be677ddd/docs/telemetry.md#errorrunlivenorecording
• https://github.com/RecordReplay/backend/blob/16a796be1a4d25f598cd17c2a77639e6be677ddd/src/instance/telemetryServer.ts#L149
• https://ui.honeycomb.io/replay/datasets/backend/result/cGjHmuMq8mg?hideMarkers

It'd be nice to also track attempts so that we can measure uptime and alert when the service is down and we stop trying

[ryanjduffy]

I think the tests are running correctly again. I'm not sure precisely what was broken but documenting what I saw and the resolution steps for posterity:

1. Running d livetest-gecko initially failed because the browser wasn't found. It was looking in .replay-playwright-browsers instead of .replay. It should have been using .replay based on my reading of the playwright installer code and the RECORD_REPLAY_DIRECTORY wasn't set to override it to something else. My next suspicion was that we had an out of date dependency. Running npm ls --depth=0 in the backend showed a lot of dependency errors so I ran npm ci to wipe and reinstall the pinned versions. This resolved finding the browser directory.
2. Once running the tests were all failing with tab crashes. I used VNC to screenshare into the macstadium box and saw several instances of replay still running (including some seemingly old ones with our old logo). I killed all of those instances and started up the livetests again (while still in VNC) and most of them appear to be running successfully. I saw a couple tab crashes but they seem to be running better now.

Going to look into adding more telemetry next so we have better insight into these failures.

[jasonLaster]

Yeah a telemetry event {name: "live-test", success: true, testName: "airbnb" }

[ryanjduffy]

Opened #89 to improve error handling. We already have telemetry for runlive tests but we lacked the test info so I've added that in https://github.com/RecordReplay/backend/pull/3198.

[ryanjduffy]

I restarted runlive on mac and we're getting the example in honeycomb now but none of the tests are passing. Investigating ...

[ryanjduffy]

The playwright browser was out of date (by a couple months) so it was unable to authenticate. Merged the latest webreplay-release into replay-playwright, ran d deploy-playwright and d release-playwright on macstadium, killed all runlive processes and open replay browsers, and started them again via d livetest-gecko and we're getting sample recordings again!

https://ui.honeycomb.io/replay/datasets/backend/result/er82JjoAcGy