search

karma-runner / karma-firefox-launcher

A Karma plugin. Launcher for Firefox.
MIT License
104 stars 54 forks source link

Firefox slow compared to Chrome #254

Closed maxs-rose closed 1 year ago

maxs-rose commented 1 year ago

Our Angular unit tests are being run on both Firefox and Chrome but Firefox is much slower than Chrome.

When running locally with ng test:

ng test --watch=false --browsers=FirefoxHeadless,ChromeHeadlessCI

✔ Browser application bundle generation complete.
10 12 2022 19:47:04.470:INFO [karma-server]: Karma v6.4.1 server started at http://localhost:9876/
10 12 2022 19:47:04.472:INFO [launcher]: Launching browsers FirefoxHeadless, ChromeHeadlessCI with concurrency 15
10 12 2022 19:47:04.475:INFO [launcher]: Starting browser Firefox
10 12 2022 19:47:04.502:INFO [launcher]: Starting browser ChromeHeadless
10 12 2022 19:47:05.266:INFO [Chrome Headless 108.0.5359.99 (Windows 10)]: Connected on socket GARFAPW9ddSN4NnQAAAB with id 93442983
10 12 2022 19:47:06.494:INFO [Firefox 107.0 (Ubuntu 0.0.0)]: Connected on socket TLWA3xgiHnlcmfIEAAAD with id 19323401
Firefox 107.0 (Ubuntu 0.0.0): Executed 1557 of 1557 SUCCESS (6 mins 15.304 secs / 4 mins 13.523 secs)
Chrome Headless 108.0.5359.99 (Windows 10): Executed 1557 of 1557 SUCCESS (1 min 7.103 secs / 1 min 2.624 secs)

Locally Firefox is installed inside WSL whilst Chrome is using the version installed in windows.

During our CI pipelines Firefox and Chrome are run separately and we are getting execution times of: Firefox: Finished in 7 mins 52.707 secs / 6 mins 29.502 secs Chrome: Finished in 3 mins 5.856 secs / 2 mins 38.854 secs

Firefox and Chrome are both running separately in Docker container for CI with a test concurrency of 1.

More recently Firefox is just disconnecting part way through the CI run causing the pipeline to fail:

09 12 2022 16:33:09.816:WARN [Firefox 102.0 (Linux x86_64)]: Disconnected (0 times) reconnect failed before timeout of 5000ms (transport close)
Firefox 102.0 (Linux x86_64) ERROR
  Disconnected reconnect failed before timeout of 5000ms (transport close)
Firefox 102.0 (Linux x86_64) ERROR
  Disconnected reconnect failed before timeout of 5000ms (transport close)
09 12 2022 16:33:09.865:INFO [karma-server]: Restarting Firefox 102.0 (Linux x86_64) (1 of 1 attempts)
09 12 2022 16:33:22.725:INFO [Firefox 102.0 (Linux x86_64)]: Disconnected browser returned on socket Op6ANd-76OD2yJGCAAAD with id 95470333.
09 12 2022 16:34:22.787:WARN [Firefox 102.0 (Linux x86_64)]: Disconnected (1 times) , because no message in 60000 ms.
Firefox 102.0 (Linux x86_64) ERROR
  Disconnected , because no message in 60000 ms.
Firefox 102.0 (Linux x86_64) ERROR
  Disconnected , because no message in 60000 ms.
Finished in 13 mins 45.687 secs / 7 mins 52.781 secs @ 16:34:22 GMT+0000 (Greenwich Mean Time)

Is there any advice to attempt to resolve this issue?

maxs-rose commented 1 year ago

I have found our issue. Turns out some of our tests were opening new tabs and as a result firefox was very aggressively throttling karma.

Fixing these tests to not actually open a new browser tab means the firefox texts are now more inline with chrome in terms to speed with firefox now only being about 40s behind chrome.

birtles commented 1 year ago

That's great news. Thank you for following up on this.