karma-runner / karma-chrome-launcher

A Karma plugin. Launcher for Chrome and Chrome Canary.
MIT License
467 stars 120 forks source link

Tests hang in Chrome and Chromium unless browser is headless or in the foreground #228

Open bard opened 3 years ago

bard commented 3 years ago

Leaving this here in case it saves someone time and headaches:

If tests are launched while Chromium is in the background, they hang. If Chromium is brought to the foreground, they resume immediately. Probably due to the tab being suspended, however passing --disable-background-timer-throttling --disable-renderer-backgrounding to the launched browser doesn't make any difference.

Tests run fine in ChromiumHeadless.

Related: https://github.com/karma-runner/karma-chrome-launcher/issues/81

rooby commented 3 years ago

I have this same issue. All my tests will throw this error: Error: Timeout - Async function did not complete within 5000ms (set by jasmine.DEFAULT_TIMEOUT_INTERVAL)

Note that when my browser is on top those errors don't appear, and they don't appear with chrome headless, so I'm fairly confident that the problem isn't just me needing to increase the timeout or fix broken tests or anything like that.

I also tried a few other Chrome flags and found that adding --disable-backgrounding-occluded-windows with the other flags helps. With those 3 flags, when my tests initially spin up they will pass without throwing the timout error, even though the browser is behind other windows in this case.

My current configuration is:

customLaunchers: {
  ChromeNoBackgroundThrottle: {
    base: 'Chrome',
    flags: [
      '--disable-background-timer-throttling',
      '--disable-renderer-backgrounding',
      '--disable-backgrounding-occluded-windows'
    ],
  }
},

It still doesn't fix the issue if the browser is actually minimised though, it's only fixed for the case where it is behind other windows.

Versions: karma@6.1.1 karma-chrome-launcher@3.1.0 Chrome: 89.0.4389.72

pfeileon commented 3 years ago

In my case it only happens very rarely. Atm I have this issue with a test for an onPush component with MatTabGroupHarness. (OnPush doesn't actually matter.)

hoangvutruong513 commented 2 years ago

I was also writing test cases for my angular app with:

  1. jasmine 3.6.0,
  2. karma 6.3.13
  3. karma-chrome-launcher 3.1.0

and encountered the very same bug, i.e. test cases hang up and fail with "Async function did not complete within 5000ms" if chrome is not the focused windows (minimized/hidden behind other windows). The very same test cases run fine in Chrome Headless Mode though.

I went through my test mocks to ensure that all my async functions did complete in time. Then tried the solution similar to @rooby, but didn't work when Chrome is actually minimized.

In the end, I basically just remove MatTabGroupHarness and MatTabModule from my Testbed import array. Then suddenly all test cases work normally even when Chrome is minimized. It could be the same root cause as @pfeileon.

My uneducated guess is that there is some hidden async/await logic in UI library like MatTabModule from Angular Material that awaits for the element to actually be rendered in Chrome (which is not done when Chrome is minimized). That is where the "Async function did not complete within 5000ms" came from.

Creating a mock for such problematic component (MatTabModule) that intentially bypass such async/await logic and declare it in your TestBed could be a potential solution, but I did not have time to test it out.