Karma sometimes generates Error: read ECONNRESET after successful test run

fr0 commented 5 years ago

Expected behaviour

This started happening intermittently after I upgrade from 3.1.3 to 3.1.4, and it has been happening ever since (not every time -- about 1/3 to 1/2 of the time). ( After looking through the changelog, I'm inclined to suspect https://github.com/karma-runner/karma/commit/cc2eff2 )

The error I get is:

$ karma start karma.conf.js --single-run --jenkins
21 02 2019 09:43:38.104:INFO [compiler.karma-typescript]: Compiling project using Typescript 3.2.4
21 02 2019 09:43:53.795:INFO [compiler.karma-typescript]: Compiled 21 files in 15446 ms.
21 02 2019 09:43:57.633:INFO [bundler.karma-typescript]: Bundled imports for 21 file(s) in 3328 ms.
21 02 2019 09:43:59.695:INFO [karma-server]: Karma v4.0.0 server started at http://0.0.0.0:9876/
21 02 2019 09:43:59.720:INFO [launcher]: Launching browsers ChromeHeadless with concurrency unlimited
21 02 2019 09:43:59.757:INFO [launcher]: Starting browser ChromeHeadless
21 02 2019 09:44:01.220:INFO [HeadlessChrome 72.0.3626 (Windows 10.0.0)]: Connected on socket 16JL3Xy0xm-ri3aBAAAA with id 59122393
HeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 0 of 110 SUCCESS (0 secs / 0 secs)
[1A[2KHeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 1 of 110 SUCCESS (0 secs / 0.061 secs)
...
[1A[2KHeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 110 of 110 SUCCESS (2.695 secs / 2.341 secs)
TOTAL: 110 SUCCESS
21 02 2019 09:44:05.251:ERROR [karma-server]: { Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:111:27) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' }
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command

Actual behaviour

Error: read ECONNRESET

Environment Details

Karma version: 4.0.1 Operating System: Windows 10 64-bit

See: https://stackoverflow.com/questions/54809981/how-do-i-debug-errors-from-karma-server

Steps to reproduce the behaviour

Unfortunately I cannot share my repo, since it is private. If someone can guide me on how to track down the source of this error, I'd be happy to do the heavy lifting.

johnjbarton commented 5 years ago

I'm surprised by this part of the output:

21 02 2019 09:44:01.220:INFO [HeadlessChrome 72.0.3626 (Windows 10.0.0)]: Connected on socket 16JL3Xy0xm-ri3aBAAAA with id 59122393
HeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 0 of 110 SUCCESS (0 secs / 0 secs)
[1A[2KHeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 1 of 110 SUCCESS (0 secs / 0.061 secs)
...
[1A[2KHeadlessChrome 72.0.3626 (Windows 10.0.0): Executed 110 of 110 SUCCESS (2.695 secs / 2.341 secs)
TOTAL: 110 SUCCESS

Is the 'progress' reporter running?

It looks like the test executes and passes, so perhaps the browser is being killed out of order withh the socket close?

Try running with logLevel debug and compare error and not-error cases.

fr0 commented 5 years ago

Sorry, the ... is just the part in the middle that I clipped out for brevity. Yes, the 'progress' reporter is running. I will try running with logLevel and get back to you. Thanks for the reply!

fr0 commented 5 years ago

Here are the logs for both the failure case and success case. failure.log success.log

johnjbarton commented 5 years ago

The key parts: FAIL : 09 04 2019 10:14:40.637:DEBUG [karma-server]: Run complete, exiting. 09 04 2019 10:14:40.638:DEBUG [launcher]: Disconnecting all browsers 09 04 2019 10:14:40.671:ERROR [karma-server]: { Error: read ECONNRESET at TCP.onStreamRead (internal/stream_base_commons.js:111:27) errno: 'ECONNRESET', code: 'ECONNRESET', syscall: 'read' } 09 04 2019 10:14:40.672:DEBUG [launcher]: Disconnecting all browsers 09 04 2019 10:14:40.674:DEBUG [launcher]: Process ChromeHeadless exited with code null and signal SIGTERM

SUCCESS: 09 04 2019 10:21:37.859:DEBUG [karma-server]: Run complete, exiting. 09 04 2019 10:21:37.860:DEBUG [launcher]: Disconnecting all browsers 09 04 2019 10:21:37.883:DEBUG [launcher]: Process ChromeHeadless exited with code null and signal SIGTERM

So it looks like the launcher kills the chrome process but the server does not know that the socket is dying on purpose. So the launcher and the server race to the next event. If the launcher wins, success. If the server wins fail.

rdrrichards commented 4 years ago

Can or will this issue be addressed at some point? Is there a work around?

johnjbarton commented 4 years ago

This issue seems to be only about an annoying and erratic error message. I suppose we should suppress the ECONNRESET error message after Run complete, exiting. Please send a PR.

mgroenhoff commented 4 years ago

@johnjbarton I sent a PR to fix this issue. This is currently making our automated tests fail because we rely on the exit code being 0.

brodziakm commented 3 years ago

Since upgrading to Angular 11 we are experiencing this problem.

I've verified a few things:

A new Angular 11 project with the default Karma test configuration does not experience this problem
Reapplying the default Karma configuration (taken from the new Angular 11 project) doesn't resolve the problem
Running a single test in our project works fine and doesn't exhibit the problem
Running a collection of tests or the whole suite exhibits the problem every single time
The attached PR resolves the issue

Will this PR be merged soon? We'd really appreciate it!

johnjbarton commented 3 years ago

Sorry which PR has a proposed fix?

rdrrichards commented 3 years ago

@johnjbarton This is the PR: https://github.com/karma-runner/karma/pull/3402

costescuandrei commented 3 years ago

Any update on this? This is still messing up the build on some comps. Maybe the root cause of this should be dealt with as well? Like making it expect the browser disconnect (not fail) or disconnects happening in the reverse order?

Panossa commented 1 year ago

End of 2022 and this is still an issue. Started in our project like a week ago. Causes our local builds to fail but runs on Jenkins without any problems.

Edit: for more context: it stopped working locally, randomly, last week. Maybe it has something to do with a Chrome update? Our Jenkins runs on an older Chrome where it still works flawlessly but locally multiple devs suddenly got problems. (Current, not working Chrome build: 106.0.5249.121. My colleague has 107.0.5304.88 and it's not working for them, either.)

karma-runner / karma