search

coder / code-server

VS Code in the browser
https://coder.com
MIT License
67.72k stars 5.56k forks source link

Suddenly stops working (TypeScript language service, terminals, extensions etc) #1912

Closed soul-codes closed 4 years ago

soul-codes commented 4 years ago

As of today, my code-server spontaneously "stopped working" in several areas -- TypeScript language features aren't working, the integrated terminal doesn't start, and none of the extensions work. Scanning through the front and backend logs makes me think that some communication issue is happening that is critical to the whole application.

Sorry if this turns out to be just some misconfiguration but as "nothing has changed" between last night when it was working properly and this morning (including configuration), I wonder if this might be a bug.

Restarting the server doesn't help. There is no difference between running it as a service with systemctl or directly executing code-server.

Reboot my host machine also doesn't help.

Logs and configs follow. <<secret:my_host>> represents a valid domain name like xyz.com that is my private development domain.

Config

bind-addr: <<secret:my_host>>:9876
auth: password
password: <<secret:password>>
cert: /etc/letsencrypt/live/<<secret:my_host>>/fullchain.pem
cert-key: /etc/letsencrypt/live/<<secret:my_host>>/privkey.pem
log: debug

Code-server stdout log

debug setting up vs code...
debug vscode got message from code-server {"message":{"type":"init","id":"YHvW1U4VeRoHIb3fgnRuDqUf","options":{"args":{"_":[],"bind-addr":"<<secret:my_host>>:9876","auth":"password","password":"39d87255b67b2e65d76705a8","cert":{"value":"/etc/letsencrypt/live/<<secret:my_host>>/fullchain.pem"},"cert-key":"/etc/letsencrypt/live/<<secret:my_host>>/privkey.pem","log":"debug","verbose":false,"config":"/root/.config/code-server/config.yaml","user-data-dir":"/root/.local/share/code-server","extensions-dir":"/root/.local/share/code-server/extensions"},"remoteAuthority":"<<secret:my_host>>:9876","startPath":{"url":"/root/code/magneto","workspace":false}}}}
debug got message from vs code {"message":{"type":"options","id":"YHvW1U4VeRoHIb3fgnRuDqUf","options":{"workbenchWebConfiguration":{"folderUri":{"$mid":1,"path":"/root/code/magneto","scheme":"vscode-remote","authority":"<<secret:my_host>>:9876"},"remoteAuthority":"<<secret:my_host>>:9876","logLevel":1,"workspaceProvider":{"payload":[["userDataPath","/root/.local/share/code-server"]]}},"remoteUserDataUri":{"$mid":1,"path":"/root/.local/share/code-server","scheme":"vscode-remote","authority":"<<secret:my_host>>:9876"},"productConfiguration":{"nameShort":"Code - OSS","nameLong":"Code - OSS","applicationName":"code-oss","dataFolderName":".vscode-oss","win32MutexName":"vscodeoss","licenseName":"MIT","licenseUrl":"https://github.com/Microsoft/vscode/blob/master/LICENSE.txt","win32DirName":"Microsoft Code OSS","win32NameVersion":"Microsoft Code OSS","win32RegValueName":"CodeOSS","win32AppId":"{{E34003BB-9E10-4501-8C11-BE3FAA83F23F}","win32x64AppId":"{{D77B7E06-80BA-4137-BCF4-654B95CCEBC5}","win32UserAppId":"{{C6065F05-9603-4FC4-8101-B9781A25D88E}","win32x64UserAppId":"{{C6065F05-9603-4FC4-8101-B9781A25D88E}","win32AppUserModelId":"Microsoft.CodeOSS","win32ShellNameShort":"C&ode - OSS","darwinBundleIdentifier":"com.visualstudio.code.oss","linuxIconName":"com.visualstudio.code.oss","licenseFileName":"LICENSE.txt","reportIssueUrl":"https://github.com/cdr/code-server/issues/new","urlProtocol":"code-oss","extensionAllowedProposedApi":["ms-vscode.vscode-js-profile-table","ms-vscode.references-view"],"builtInExtensions":[{"name":"ms-vscode.node-debug","version":"1.44.5","repo":"https://github.com/Microsoft/vscode-node-debug","metadata":{"id":"b6ded8fb-a0a0-4c1c-acbd-ab2a3bc995a6","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.node-debug2","version":"1.42.2","repo":"https://github.com/Microsoft/vscode-node-debug2","metadata":{"id":"36d19e17-7569-4841-a001-947eb18602b2","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.references-view","version":"0.0.51","repo":"https://github.com/Microsoft/vscode-reference-view","metadata":{"id":"dc489f46-520d-4556-ae85-1f9eab3c412d","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.js-debug-companion","version":"1.0.0","repo":"https://github.com/microsoft/vscode-js-debug-companion","metadata":{"id":"99cb0b7f-7354-4278-b8da-6cc79972169d","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.js-debug-nightly","version":"2020.4.2917","repo":"https://github.com/Microsoft/vscode-js-debug","metadata":{"id":"7acbb4ce-c85a-49d4-8d95-a8054406ae97","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.vscode-js-profile-table","version":"0.0.2","repo":"https://github.com/Microsoft/vscode-js-debug","metadata":{"id":"7e52b41b-71ad-457b-ab7e-0620f1fc4feb","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}}],"commit":"48f7c2724827e526eeaa6c2c151c520f48a61259","date":"2020-06-04T10:39:14Z","version":"1.45.1","extensionsGallery":{"serviceUrl":"https://extensions.coder.com/api","itemUrl":"","controlUrl":"","recommendationsUrl":""}},"nlsConfiguration":{"locale":"en","availableLanguages":{}},"commit":"48f7c2724827e526eeaa6c2c151c520f48a61259"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"Management","reconnectionToken":"65c32862-1c78-422f-8176-13f14559e9e9","reconnection":"false","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"a9e8ccab-631b-493b-ae75-14e0757a7c86","reconnection":"false","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"a9e8ccab-631b-493b-ae75-14e0757a7c86","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"a9e8ccab-631b-493b-ae75-14e0757a7c86","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"a9e8ccab-631b-493b-ae75-14e0757a7c86","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"a9e8ccab-631b-493b-ae75-14e0757a7c86","reconnection":"true","skipWebSocketFrames":"false"}}}
debug setting up vs code...
debug vscode got message from code-server {"message":{"type":"init","id":"QzLmGrqMypsmjxbTG8xbS0qY","options":{"args":{"_":[],"bind-addr":"<<secret:my_host>>:9876","auth":"password","password":"39d87255b67b2e65d76705a8","cert":{"value":"/etc/letsencrypt/live/<<secret:my_host>>/fullchain.pem"},"cert-key":"/etc/letsencrypt/live/<<secret:my_host>>/privkey.pem","log":"debug","verbose":false,"config":"/root/.config/code-server/config.yaml","user-data-dir":"/root/.local/share/code-server","extensions-dir":"/root/.local/share/code-server/extensions"},"remoteAuthority":"<<secret:my_host>>:9876","startPath":{"url":"/root/code/magneto","workspace":false}}}}
debug got message from vs code {"message":{"type":"options","id":"QzLmGrqMypsmjxbTG8xbS0qY","options":{"workbenchWebConfiguration":{"folderUri":{"$mid":1,"path":"/root/code/magneto","scheme":"vscode-remote","authority":"<<secret:my_host>>:9876"},"remoteAuthority":"<<secret:my_host>>:9876","logLevel":1,"workspaceProvider":{"payload":[["userDataPath","/root/.local/share/code-server"]]}},"remoteUserDataUri":{"$mid":1,"path":"/root/.local/share/code-server","scheme":"vscode-remote","authority":"<<secret:my_host>>:9876"},"productConfiguration":{"nameShort":"Code - OSS","nameLong":"Code - OSS","applicationName":"code-oss","dataFolderName":".vscode-oss","win32MutexName":"vscodeoss","licenseName":"MIT","licenseUrl":"https://github.com/Microsoft/vscode/blob/master/LICENSE.txt","win32DirName":"Microsoft Code OSS","win32NameVersion":"Microsoft Code OSS","win32RegValueName":"CodeOSS","win32AppId":"{{E34003BB-9E10-4501-8C11-BE3FAA83F23F}","win32x64AppId":"{{D77B7E06-80BA-4137-BCF4-654B95CCEBC5}","win32UserAppId":"{{C6065F05-9603-4FC4-8101-B9781A25D88E}","win32x64UserAppId":"{{C6065F05-9603-4FC4-8101-B9781A25D88E}","win32AppUserModelId":"Microsoft.CodeOSS","win32ShellNameShort":"C&ode - OSS","darwinBundleIdentifier":"com.visualstudio.code.oss","linuxIconName":"com.visualstudio.code.oss","licenseFileName":"LICENSE.txt","reportIssueUrl":"https://github.com/cdr/code-server/issues/new","urlProtocol":"code-oss","extensionAllowedProposedApi":["ms-vscode.vscode-js-profile-table","ms-vscode.references-view"],"builtInExtensions":[{"name":"ms-vscode.node-debug","version":"1.44.5","repo":"https://github.com/Microsoft/vscode-node-debug","metadata":{"id":"b6ded8fb-a0a0-4c1c-acbd-ab2a3bc995a6","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.node-debug2","version":"1.42.2","repo":"https://github.com/Microsoft/vscode-node-debug2","metadata":{"id":"36d19e17-7569-4841-a001-947eb18602b2","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.references-view","version":"0.0.51","repo":"https://github.com/Microsoft/vscode-reference-view","metadata":{"id":"dc489f46-520d-4556-ae85-1f9eab3c412d","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.js-debug-companion","version":"1.0.0","repo":"https://github.com/microsoft/vscode-js-debug-companion","metadata":{"id":"99cb0b7f-7354-4278-b8da-6cc79972169d","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.js-debug-nightly","version":"2020.4.2917","repo":"https://github.com/Microsoft/vscode-js-debug","metadata":{"id":"7acbb4ce-c85a-49d4-8d95-a8054406ae97","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}},{"name":"ms-vscode.vscode-js-profile-table","version":"0.0.2","repo":"https://github.com/Microsoft/vscode-js-debug","metadata":{"id":"7e52b41b-71ad-457b-ab7e-0620f1fc4feb","publisherId":{"publisherId":"5f5636e7-69ed-4afe-b5d6-8d231fb3d3ee","publisherName":"ms-vscode","displayName":"Microsoft","flags":"verified"},"publisherDisplayName":"Microsoft"}}],"commit":"48f7c2724827e526eeaa6c2c151c520f48a61259","date":"2020-06-04T10:39:14Z","version":"1.45.1","extensionsGallery":{"serviceUrl":"https://extensions.coder.com/api","itemUrl":"","controlUrl":"","recommendationsUrl":""}},"nlsConfiguration":{"locale":"en","availableLanguages":{}},"commit":"48f7c2724827e526eeaa6c2c151c520f48a61259"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"Management","reconnectionToken":"480f9d98-d140-4e7a-947b-980e640d790f","reconnection":"false","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"false","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}

debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}
debug vscode got message from code-server {"message":{"type":"socket","query":{"type":"ExtensionHost","reconnectionToken":"e9f903ca-2d15-40c6-b86e-0af1c02f45db","reconnection":"true","skipWebSocketFrames":"false"}}}

Chrome log

[Service Worker] registered
log.ts:179 DEBUG Checking for update...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>.me:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
extensionHostProcessManager.ts:73 Error received from starting extension host
(anonymous) @ extensionHostProcessManager.ts:73
Promise.then (async)
e @ extensionHostProcessManager.ts:68
_createInstance @ instantiationService.ts:111
createInstance @ instantiationService.ts:76
_createExtensionHosts @ extensionService.ts:97
_startExtensionHostProcess @ abstractExtensionService.ts:135
_initialize @ abstractExtensionService.ts:100
e @ extensionService.ts:59
_createInstance @ instantiationService.ts:111
_createServiceInstance @ instantiationService.ts:210
_createServiceInstanceWithOwner @ instantiationService.ts:199
_createAndCacheServiceInstance @ instantiationService.ts:188
_getOrCreateServiceInstance @ instantiationService.ts:136
get @ instantiationService.ts:54
(anonymous) @ workbench.ts:141
invokeFunction @ instantiationService.ts:61
startup @ workbench.ts:137
open @ web.main.ts:97
async function (async)
open @ web.main.ts:68
t.main @ web.main.ts:362
t.create @ workbench.web.api.ts:318
(anonymous) @ workbench.ts:336
(anonymous) @ workbench.ts:286
t._invokeFactory @ loader.js:1043
t.complete @ loader.js:1053
s._onModuleComplete @ loader.js:1674
s._resolve @ loader.js:1634
s.defineModule @ loader.js:1281
r @ loader.js:1724
(anonymous) @ util.ts:12
(anonymous) @ fake:1
extensionHostProcessManager.ts:74 timeout
(anonymous) @ extensionHostProcessManager.ts:74
Promise.then (async)
e @ extensionHostProcessManager.ts:68
_createInstance @ instantiationService.ts:111
createInstance @ instantiationService.ts:76
_createExtensionHosts @ extensionService.ts:97
_startExtensionHostProcess @ abstractExtensionService.ts:135
_initialize @ abstractExtensionService.ts:100
e @ extensionService.ts:59
_createInstance @ instantiationService.ts:111
_createServiceInstance @ instantiationService.ts:210
_createServiceInstanceWithOwner @ instantiationService.ts:199
_createAndCacheServiceInstance @ instantiationService.ts:188
_getOrCreateServiceInstance @ instantiationService.ts:136
get @ instantiationService.ts:54
(anonymous) @ workbench.ts:141
invokeFunction @ instantiationService.ts:61
startup @ workbench.ts:137
open @ web.main.ts:97
async function (async)
open @ web.main.ts:68
t.main @ web.main.ts:362
t.create @ workbench.web.api.ts:318
(anonymous) @ workbench.ts:336
(anonymous) @ workbench.ts:286
t._invokeFactory @ loader.js:1043
t.complete @ loader.js:1053
s._onModuleComplete @ loader.js:1674
s._resolve @ loader.js:1634
s.defineModule @ loader.js:1281
r @ loader.js:1724
(anonymous) @ util.ts:12
(anonymous) @ fake:1
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] waiting for 5 seconds before reconnecting...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] resolving connection...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] connecting to <<secret:my_host>>:9876...
log.ts:185  INFO [remote-connection][ExtensionHost][e9f90…][reconnect] reconnected!
soul-codes commented 4 years ago

I can confirm that a fresh install on a completely newly spun server on DigitalOcean reproduces this issue.

soul-codes commented 4 years ago

For a completely mysterious reason, it "suddenly works" again, so I guess this could have been seen as an Act of God or perhaps a cloud provider's temporary network issue of some kind. In any case, I am no longer personally affected by this issue, but I'll leave it open in case the logs above hint at any genuine bug.

code-asher commented 4 years ago

It might not have been a network issue since it looks like only the extension host is trying to reconnect while the other socket appears stable. It's possible the extension host was crashing or exiting for some reason. There might be more information that explains why in VS Code's logs which are in ~/.local/share/code-server/logs/[date] by default.

I'll close for now since it's working for you and I don't see anything actionable in the logs, but if it happens again or if you still have those logs definitely feel free to post them here!