search

defenseunicorns / leapfrogai

Production-ready Generative AI for local, cloud native, airgap, and edge deployments.
https://leapfrog.ai
Apache License 2.0
249 stars 26 forks source link

chore(ui): e2es w/multiple browsers #742

Open andrewrisse opened 1 month ago

andrewrisse commented 1 month ago

We are currently running our UI Playwright e2e tests only in Chrome due to performance/flakiness issues.

Running the tests concurrently (disabling parallelism), seems to allow the tests to pass, although it is very slow.

There seems to be two issues:

  1. The API (especially when attaching files to assistants) slows down too much when multiple browsers are running multiple tests against it. This causes lots of test flakiness and failures. I'm hoping when we handle concurrent requests better, we can re-enable more browsers.
  2. A couple tests can cause issues if two browsers run them at the same time. For example, I noticed one of our tests specifically checks navigation history, and multiple browsers running this test at the same time causes flakiness. I didn't think the different browsers would affect the navigation history, but I saw this issue consistently and ran out of time to debug.

Investigate these issues further to confirm root causes and create issues and/or fix so the tests can run in parallel, on all browsers, headed and headless, on a local machine and in the pipeline.

andrewrisse commented 1 month ago

blocked until we have bigger runners