jtsom
commented
7 years ago I'm seeing the same thing. Mine is on Windows 10, and when it happens, VS Code locks up and take the whole Window system with it.
Closed jens1o closed 7 years ago
jtsom
commented
7 years ago I'm seeing the same thing. Mine is on Windows 10, and when it happens, VS Code locks up and take the whole Window system with it.
jens1o
commented
7 years ago The Insiders version is unuseable anymore, because it looks like even the buillt-in extensions are limited in their functionality. I expected a fix in todays build, but there is none. 😭
jtsom
commented
7 years ago The TypeScript Hero extension causes the extension host to blow up spectacularly.
index.js:44 Error: read ECONNRESET
at exports._errnoException (util.js:1050)
at Pipe.onread (net.js:581)
onError @ index.js:44
(anonymous) @ index.js:110
emitOne @ events.js:96
emit @ events.js:191
window.onerror @ /usr/share/code-insiders/resources/electron.asar/renderer/init.js:135
i.globals.onerror @ errorTelemetry.ts:64
events.js:163 Uncaught Error: read ECONNRESET
at exports._errnoException (util.js:1050)
at Pipe.onread (net.js:581)
exports._errnoException @ util.js:1050
onread @ net.js:581
messageService.ts:125 Extension host terminated unexpectedly. Please reload the window to recover.
e.doShow @ messageService.ts:125
e.show @ messageService.ts:104
t.onExit @ extensionHost.ts:380
(anonymous) @ extensionHost.ts:196
emitTwo @ events.js:106
emit @ events.js:194
__dirname.ChildProcess._handle.onexit @ internal/child_process.js:215
extensionHost.ts:388 Extension host terminated unexpectedly. Code: 87 Signal: null
t.onExit @ extensionHost.ts:388
(anonymous) @ extensionHost.ts:196
emitTwo @ events.js:106
emit @ events.js:194
__dirname.ChildProcess._handle.onexit @ internal/child_process.js:215
extensionHost.ts:180 Extension Host
extensionHost.ts:181 The module native-watchdog has detected that the event loop is unresponsive.
Here is the JavaScript stack trace:
======================================native-watchdog======================================
Object.normalizePath (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:2762:49)
createSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16645:38)
parseSourceFileWorker (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16572:26)
Object.parseSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16521:26)
Object.createSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16371:29)
filePathes.map.o (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:0:0)
TypescriptParser.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:1:1)
/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:96:75
Object.__awaiter (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:92:16)
TypescriptParser.parseFiles (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:66:24)
DeclarationIndex.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/DeclarationIndex.js:1:1)
/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:96:75
Object.__awaiter (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:92:16)
DeclarationIndex.buildIndex (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/DeclarationIndex.js:133:24)
ImportResolveExtension.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/out/src/extension/extensions/ImportResolveExtension.js:1:1)
fulfilled (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:1:1)
process._tickCallback (internal/process/next_tick.js:109:7)
======================================native-watchdog======================================
The module native-watchdog will now terminate the process with exit code 87.
Personally, I don't have this extension installed :/
jtsom
commented
7 years ago I don't anymore either!
jens1o
commented
7 years ago Does that fix it for you?
jtsom
commented
7 years ago Well, the crash doesn't point to that specific extension now - but I've had it show the extension host is dead message without it. I've had to trim down my extensions.
jens1o
commented
7 years ago Okay, I have the same at my machine...
jens1o
commented
7 years ago oops.
bk147
commented
7 years ago In my experience some Extensions result in "Extension host terminated..." - in my case "Visual Studio Team Services 1.122.0" and sometimes "Git Lens... 4.3.3". Disabling both of these and the Error disappears (I have a number of other extensions working :-))
jtsom
commented
7 years ago If I leave my system idle, when I come back to it I see the host terminated message and this in the dev. console:
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28 [Embedded Page] Refused to load the image 'http://g.recordit.co/2yRF1XeSbv.gif' because it violates the following Content Security Policy directive: "img-src https:".
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28 Extension host terminated unexpectedly. Please reload the window to recover.
e.doShow @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29 Extension host terminated unexpectedly. Code: 3221225477 Signal: null
t.onExit @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29But this only started back yesterday, and there wasn't any update to these extensions. Earlier, they worked like a charm.
jtsom
commented
7 years ago Yes. Whatever changes were made to the extension host process that landed yesterday (in commit baa6fbece9035e3d20cbcb776d73ebc6c4ddde26) is causing these issues. (If you look at the commits that would have been in that release you'll see several dealing with Extension Host)
iursevla
commented
7 years ago Had this problem but i think it was because i block VSCode Insiders from fetching a new version everyday (aka Firewall blocked with Glasswire), because i'm on a slow connection and downloading 60MB everyday makes a difference.
Liquidmantis
commented
7 years ago I'm running into this as well. I was going through a binary search pattern of disabling extensions but it seems inconsistent. Seems okay on Mac though, Windows is where I'm seeing it.
jtsom
commented
7 years ago I've seen it on the Mac and Windows and even Ubuntu Linux - especially with Typescript Hero extension.
thewashingtonhua
commented
7 years ago I'm on macOS. it happened this morning. first in 1.15.1. and I upgraded to 1.16.0, didn't solve.
I looked into the dev tool and it mentioned some extensions' name. I tried disable them. but nothing changed after I disabled plenty of them. so it had be a problem of vscode insider itself.
for now I switched back to stable release so I can continue my work today. waiting for the team to fix this ASAP.
jens1o
commented
7 years ago @alexandrudima Did you have a chance to look at this?
jtsom
commented
7 years ago I just launched VC Code on my mac, switched to my email for a second, and when I switched back I had the extension host crashed message. The developer tools shows this: (it looks like the error message is a little more descriptive, but a pretty error message is still an error):
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:29 Extension Host
/Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:29 The module native-watchdog has detected that the event loop is unresponsive.
Here is the JavaScript stack trace:
======================================native-watchdog======================================
Timer.listOnTimeout (timers.js:162:23)
======================================native-watchdog======================================
The module native-watchdog will now terminate the process with exit code 87.
Another data point. I just launched the app on Ubuntu Linux, and almost immediately received:
index.js:5 Error: read ECONNRESET
at exports._errnoException (util.js:1050:11)
at Pipe.onread (net.js:581:26)
onError @ index.js:5
events.js:163 Uncaught Error: read ECONNRESET
at exports._errnoException (util.js:1050:11)
at Pipe.onread (net.js:581:26)
/usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:28 Extension host terminated because it was not responsive. Please reload the window to recover.
e.doShow @ /usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:28
/usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:29 Extension host terminated unexpectedly. Code: 87 Signal: null
t.onExit @ /usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:29
/usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:29 Extension Host
/usr/share/code-insiders/resources/app/out/vs/workbench/workbench.main.js:29 The module native-watchdog has detected that the event loop is unresponsive.
Here is the JavaScript stack trace:
======================================native-watchdog======================================
Object.<anonymous> (/usr/share/code-insiders/resources/app/extensions/git/out/commands.js:1:1)
Object.<anonymous> (/usr/share/code-insiders/resources/app/extensions/git/out/commands.js:1205:3)
Module._compile (module.js:571:32)
Object.Module._extensions..js (module.js:580:10This is version Version 1.16.0-insider Commit 9e7441df9f609500a13e8651bd01eccf1613f386 Date 2017-08-17T05:16:03.108Z Shell 1.7.4 Renderer 58.0.3029.110 Node 7.9.0 Architecture x64
jtsom
commented
7 years ago After waking the system:
Extension host terminated unexpectedly. Please reload the window to recover.
e.doShow @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29 Extension host terminated unexpectedly. Code: 3221225477 Signal: null
t.onExit @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29 net::ERR_NETWORK_CHANGED
Error: net::ERR_NETWORK_CHANGED: Error: net::ERR_NETWORK_CHANGED
Error: net::ERR_NETWORK_CHANGED
at Timeout._onTimeout (C:\Program Files\Microsoft VS Code Insiders\resources\app\out\vs\code\electron-main\main.js:4:78040)
at ontimeout (timers.js:386:14)
at tryOnTimeout (timers.js:250:5)
at Timer.listOnTimeout (timers.js:214:5)
Tyriar
commented
7 years ago I've been seeing this a lot on macOS.
Tyriar
commented
7 years ago Also the message box ignores escape and will not go away until you reload the window.
jtsom
commented
7 years ago Another after wake from sleep:
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29 invalid provider: Error: invalid provider
at e.setActiveSCMProvider (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:2791025)
at e.set [as activeProvider] (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:2790396)
at file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:2791728
at Object.dispose (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:4:136249)
at file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:3160816
at Array.forEach (native)
at e.dispose (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:3160758)
at e._stopExtensionHostProcess (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:3180051)
at e._onExtensionHostCrashed (file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:3181277)
at file:///C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28:3180458
e.onUnexpectedError @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:29
/C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28 Extension host terminated unexpectedly.
e.doShow @ /C:/Program Files/Microsoft VS Code Insiders/resources/app/out/vs/workbench/workbench.main.js:28Finally a reaction from the vscode team... Thanks @Tyriar 💓
jens1o
commented
7 years ago Also the message box ignores escape and will not go away until you reload the window.
That's a bug? I thought this is by design because the extension host is quite critical.
alexdima
commented
7 years ago Some context on what has changed in the last few days in the Insiders.
As part of working on #26445 i.e. If an extension would run into a condition which would make it execute forever (i.e. an unintentional while(true) loop), the extension host process would never terminate and would continue running at 100% CPU until it was manually killed or until the system was restarted.
I've implemented a native node module that uses a C++ thread (https://github.com/Microsoft/node-native-watchdog) that executes in the extension host process. If the JavaScript event loop thread (the main nodejs loop where all the extensions execute) is busy for more than 10s, the C++ thread will terminate the extension host process with exit code 87 and print to stderr the fact that it did so (it will identify itself), and will also attempt to extract the JavaScript stack trace that was causing the stall.
The watchdog mechanism had a simple, but important flaw - #32620. It was using absolute epoch time, and, when waking up the laptop from sleep, if the JavaScript event loop would execute first, all would be fine, if the C++ thread would execute first, it would believe that the JavaScript event loop was busy for the amount of time the computer was in sleep and it would incorrectly terminate the process. I've addressed this flaw and the Insiders build on Monday will contain the fix. Sorry about it!
There might be other flaws in this watchdog code, and if we discover them, we will fix them, or if we fail to do so, we will remove the mechanism and go back to leaking 100% CPU processes.
That being said, we have also recently put the 64 bit version of VS Code on Windows into spotlight. I don't believe any of us have explicitly tested what happens when an extension that ships native node modules compiled against Electron 32 bit will try to load up those .node files (i.e. DLL files) on the 64 bit version of VS Code. I'm mentioning this because we've received more reports on crashes on Windows 64 bit even on the stable version, which does not contain the above mentioned watch dog mechanism.
I will try to cover everyone:
@jens1o Can you please click on the button "Developer Tools" and let us know of any stderr output present there and the exit code of the extension host process?
@jtsom IMHO this is a great find.
The extension rbbit.typescript-hero-1.4.0 keeps the extension host busy for more than 10s. That means all other extensions installed on your machine will not do anything while that extension executes and that you have been having a very bad VS Code experience since you've installed this extension. This looks very worry-some, this extension appears to load the entire TypeScript language service in the extension host process and compute very heavy stuff, blocking everyone else. Could you please reach out to the extension author and point out that this is an anti-pattern? For example, in our built-in typescript extension, we also load the Typescript language service, but we do so in a separate process explicitly in order to not block any other extension execution and have a good end-user experience.
The module native-watchdog has detected that the event loop is unresponsive.
Here is the JavaScript stack trace:
======================================native-watchdog======================================
Object.normalizePath (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:2762:49)
createSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16645:38)
parseSourceFileWorker (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16572:26)
Object.parseSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16521:26)
Object.createSourceFile (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript/lib/typescript.js:16371:29)
filePathes.map.o (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:0:0)
TypescriptParser.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:1:1)
/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:96:75
Object.__awaiter (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:92:16)
TypescriptParser.parseFiles (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/TypescriptParser.js:66:24)
DeclarationIndex.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/DeclarationIndex.js:1:1)
/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:96:75
Object.__awaiter (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:92:16)
DeclarationIndex.buildIndex (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/typescript-parser/DeclarationIndex.js:133:24)
ImportResolveExtension.<anonymous> (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/out/src/extension/extensions/ImportResolveExtension.js:1:1)
fulfilled (/home/john/.vscode-insiders/extensions/rbbit.typescript-hero-1.4.0/node_modules/tslib/tslib.js:1:1)
process._tickCallback (internal/process/next_tick.js:109:7)
======================================native-watchdog======================================
The module native-watchdog will now terminate the process with exit code 87.Your second encounter seems to have been caused by the sleep flaw in the watchdog -- #32620 -- a fix will come out on Monday.
Your third encounter on Ubuntu seems to point to our built-in git extension. If you can consistently reproduce, could you please open a separate issue and tell @joaomoreno as many details about your setup as possible so we can track down why the git extension would block for such a long time the event loop.
@bk147 @iursevla @Liquidmantis @tonghuashuo If you ever see the extension host crashing, please check for the stderr and exit code in the Developer Tools. We'd like to know why it has crashed.
Thank you for all your help in troubleshooting this. It might turn out that we cannot ship a watchdog or perhaps not in its current form. I am an optimist, and I hope that fixing the sleeping flaw will reduce the rate of false-positives and begin to reveal the extensions that have flaws which lead to long stalls and effectively ruin your VS Code experience. Once again, sorry about the sleeping flaw in the watchdog!
jens1o
commented
7 years ago @alexandrudima Sure, here you go:
Code: 3221225477
But it seems to be very generic...
alexdima
commented
7 years ago @jens1o Thank you. I think that would be 0xC0000005 -- Access Violation -- Window's version of "segmentation fault".
Do you get the access violation on startup, 10s after startup, when waking up the laptop, when doing something specific ?
jens1o
commented
7 years ago I just open Visual Studio Code (Insiders). It takes some seconds, but that didn't changed... It always took some seconds so the first extension services popped up. Around that time, I get this error. But it only started once you made some major changes to the host itself.
Liquidmantis
commented
7 years ago @alexandrudima First, thanks for the detailed writeup! I've been seeing the same vague error as jens1o. On my below screencap the first two errors matching jen1o are a few seconds after opening VSCode Insiders x64. The next two more detailed errors are after trying the new Restart Extension Host option.
Tyriar
commented
7 years ago FYI I seem to see it every time I wake my mac up (it happens on all Windows).
Liquidmantis
commented
7 years ago Unfortunately, today's update, 0626479, behaves no differently. Sounds like that was probably expected though since my problem occurs at launch time and not on wake.
jens1o
commented
7 years ago Yeah, no different at my machine too.
jens1o
commented
7 years ago Does someone of the vscode team actually works on this issue? Personally, I would have expected a fix as soon as possible, while it is labeled as "important".... 😭
I'm waiting for one week now... 😢
SQLDBAWithABeard
commented
7 years ago I too get this - completely random - sometimes restart extension host works sometimes it doesn't.
Always on starting the application (latest insiders version)
jens1o
commented
7 years ago WHAT DID YOU GUYS DO?!
THANKS THANKS THANKS!!!! IT'S WORKING FINE <3333333333
THANK YOU VERY MUCH :)
IT'S FIXED!!!!
(sorry for capslock) 😆
jtsom
commented
7 years ago Guessing from this commit, that looks to be included in the latest release: https://github.com/Microsoft/vscode/commit/55413ced7604f34c5360f7b6ad3f30be1aaca38b
"Remove watchdog for now"
alexdima
commented
7 years ago Due to too much instability, with both upgrading to Electron 1.7.5 (see the critical issue #33016), and the watchdog which definitely has some sort of flaw (only on Windows), we've removed the watchdog for now (see #26445). I will try to add the watchdog again only after figuring out what causes the ACCESS VIOLATION.
Again, I'm sorry for the temporary extra instability caused by the watchdog.
jens1o
commented
7 years ago Thank you :)
SQLDBAWithABeard
commented
7 years ago That appears to have done the trick here. Thank you
SteveStrongApp
commented
6 years ago Sorry guys this problem is back, installing 64 bit version on win 10 machine. I do not have full admin rights. working in corporate environment. what do I need to help you fix this problem
alexdima
commented
6 years ago @SteveStrongBAH please open a separate issue with as much information as possible.
(2 theme extensions excluded)
Steps to Reproduce:
Since I profiled startup this morning, the extension host is dead :/
Reproduces without extensions: No