Open adrianghc opened 4 years ago
@aeschli Is this just another variation of challenges with reconnecting after sleep?
Seems like the WSL remote extension is using TCP/IP for the connectivity to the WSL 2 container. When the system enters modern standby, Hyper-V will pause running VMs and Hyper-V-VmSwitch
will "disconnect" itself from host. On a system with plenty of virtual switches, Hyper-V will sometimes takes a full minute to restore connectivity of all of those virtual switches, further exaggerating this problem.
Replacing TCP/IP sockets with the Hyper-V sockets (AF_VSOCK
on the Linux side and AF_HYPERV
on the Windows side) instead of the regular TCP/IP should solve this problem.
Let see if increasing the timeout helps
I wasn't able to reproduce the issue.
Lately I have found that this issue does not always occur, or at least it seems to happens less frequently. Perhaps something seemingly unrelated somehow mitigated this to a degree. I'll try to see if I find a pattern, and will try to get a better idea of how often it still occurs.
Issue happened (much) less often if I can get Hyper-V virtual switches to wake up faster, in case of short modern standby sessions. (e.g. the typical duration of lunch break, commuting, etc.). Several ways that I can do to speed this up are:
NetSetupSvc
after WSL 2 is started.Keeping the system for an extended amount in modern standby (e.g. overnight) will almost certainly reproduce this behavior. Most of the time it will also result in VSCode asking for "reload the window" as well (like in #3126).
Therefore, I really hope that you migrate to AF_VSOCK
/AF_HYPERV
instead. Or perhaps, make this extension open source so I can modify it. 😁
Yeah this problem is horrible please escalate. I find myself having to restart vscode 2x to 3x daily because I do all my work in WSL remote and the reload ruins my terminal sessions with running programs. Terrible for programmer productivity.
Also for whatever stupid clearly related reason workplace settings on remote are not honored. So my windows always lose their color scheme (I use a different color scheme for each project). I'll open a second ticket for this issue but mention it because this chore of manually resetting color schemes in 6 windows wouldn't be relevant except for this super weird reload bug.
Please fix it ASAP thanks.
I'd add that once WSL 2 is started, the hns
service can be disabled to prevent the Hyper-V virtual switches from disconnecting when the computer enters modern standby. On my Dell XPS 15 9560, disabling this service doesn't seem to prevent the computer from entering DRIPS either.
The only downside is that if we stop the WSL 2 (e.g. by logging off or running wsl --shutdown
), it won't be able to start. In that case, to restart WSL 2 without rebooting the computer, we can take the following steps in order:
LxssManager
service.hns
service.hns
service.Had done this in the last 4 days and no more reconnecting / "please reload window" after resuming from modern standby, even when the modern standby lasted overnight. It now works flawlessly as it should. But I have to stress again, this is just a hack, and there should be no reason for VSCode not to use AF_HYPERV
/AF_VSOCK
.
Also to Hyper-V team (where I can file this to them by the way?), there should be no reason why would you disconnect the virtual switches when the computer enters modern standby. I didn't find any single issue so far, even with regular Hyper-V VMs. Both the default NAT network and bridged networks works flawlessly as well with the hns
service disabled after I've started at least one VM that uses the virtual switches.
For me the issue started when I installed 2 extensions on VSCode to the point that VSCode would just work for 2-3 seconds only. After deleting them, it is working so far. Extensions are:
I wonder if this is happening because of clock drift, especially after hibernation? I always get this "Cannot reconnect" from VS Code after waking my PC from hibernation, and there are known problems with the WSL clock not re-syncing with the system clock: https://github.com/microsoft/WSL/issues/5324
I wonder if this is happening because of clock drift, especially after hibernation? I always get this "Cannot reconnect" from VS Code after waking my PC from hibernation, and there are known problems with the WSL clock not re-syncing with the system clock: microsoft/WSL#5324
I don't think so. This happens to my system with hybernation completely turned off (powercfg -h off
) during modern standby. And clock drifts didn't happen at all under modern standby AFAIK, at least on my systems. Further, disabling the hns
service (in my previous post) pretty much solved the problem. After disabling the hns
service, sometimes, VSCode's remote extension do still timeout, but reconnecting happens instantly (e.g. in a blink of an eye), and I never been asked to reload the Window.
Now I am running Insider build 20279, and this problem is mostly gone (much less frequent compared to in 19041). However, disabling the hns
service still yields the best result.
Seems related to https://github.com/microsoft/vscode-remote-release/issues/3158 and https://github.com/microsoft/vscode-remote-release/issues/3126.
This is incredibly annoying, because the only way I've found to break the reload cycle is to reload the window, sit there waiting for it to pop up with the 5 second reconnect countdown, and then interrupt that 5 second countdown by clicking the "reload window" button on that message. If I miss that 5 second interval and let it try to reconnect on its own, it will fail again, and the cycle starts over.
This is incredibly annoying, because the only way I've found to break the reload cycle is to reload the window, sit there waiting for it to pop up with the 5 second reconnect countdown, and then interrupt that 5 second countdown by clicking the "reload window" button on that message. If I miss that 5 second interval and let it try to reconnect on its own, it will fail again, and the cycle starts over.
@tristanbrown As a workaround, you can always do F1 > Developer: Reload Window
. You could even assign a keybinding to it so you don't need to wait.
The "Reload Window" button is shown all the time once the modal dialog comes in:
But @aeschli I miss the context here. Is it necessary to do two reloads for the window to recover in this case ?
A single reload does the trick, right @dbaeumer ?
Right, a single Windows reload makes it work for me again.
This issue started happening for me too since lately. I don't know whether it's due to a Windows update or an extension update. I put my PC in hybernation overnight.
It used to work flawlessly.
- Stop the
LxssManager
service.- Enable and start the
hns
service.- Start the WSL 2 using the regular means.
- Stop and disable the
hns
service.
@jamestut How do you stop LxssManager and enable/disable hns
?
Guys, I think my issue #4353 may be related to this. I hibernate my computer every night. I just restarted my computer and could not reproduce it, while it happened every time before the restart. I am pretty sure I restarted yesterday because of an update so I think 1 hibernation is enough to trigger the problem. I guess after hibernating tonight it will start happening again.
I wonder if something happens after hibernation which makes reconnection not work at all? The reproduction would be:
@aeschli, is there a way we can assist in fixing this?
Update: After 6 hours (11AM to 5PM) of hibernation the problem does not occur. Will see tomorrow morning.
I have recently pushed a change which might improve things in this area -- https://github.com/microsoft/vscode/commit/32d29d71262ce330097e1f9d826344912d6203ad
Basically, if the vscode renderer would disconnect from the server before the laptop went to sleep (let's say at 10pm), and then the laptop would be opened in the morning (let's say at 8am), then the reconnection loop would attempt exactly once to reconnect. If WSL would be fast enough to restore the network port forwarding, then the reconnection would succeed on the first try. But if WSL would be a bit slow in restoring the network port forwarding, then the reconnection loop would give up after a single attempt, because it would think that more than 3hrs elapsed and the server would not wait so long for a reconnection.
I have changed the logic to attempt to reconnect 360 times before giving up instead of 3hrs. The original idea was to have some kind of stop mechanism eventually, and trying multiple times (with a counted limit instead of a time limit) has the same characteristic.
So I'd like to ask if anyone that is using VS Code Insiders is still seeing issues in this area. The change has not yet made it to stable, but will be a part of our upcoming 1.58 release
I have recently pushed a change which might improve things in this area -- microsoft/vscode@32d29d7 … So I'd like to ask if anyone that is using VS Code Insiders is still seeing issues in this area. The change has not yet made it to stable, but will be a part of our upcoming 1.58 release
@alexdima I think your fix targets only one error in this complex bug: the reconnection timeout. I was experiencing this issue (more precisely, #3126 since I experience this issue on both WSL 1 and WSL 2 and my my device does not support Modern Standby) After the fix the issue, I am still experiencing it (at least on WSL 1, not tested on WSL 2). But now if I dig deep into the logs, I noticed that the logged error messages have changed.
Before the fix 32d29d7
my logs are posted in https://github.com/microsoft/vscode-remote-release/issues/3126#issuecomment-807042511.
For convenience I will repost the download links to the logs before the fix here:
My log after the fix is here: wsl1-32d29d7-fix.log
Notice the date changed from 08-08
to 08-09
as I left my laptop sleep overnight, and I was greeted with the error in the morning.
The Unknown reconnection token
error in the log leads me to this issue comment: https://github.com/microsoft/vscode-remote-release/issues/1616#issuecomment-686788922
I would be happy to give out more diagnostic information if necessary.
This is incredibly annoying. I tried the disable hns
service workaround but it does nothing to stop the issue from happening.
I keep getting this annoying screen that I cannot bypass nondestructively:
There is no "Reconnect Now" button. I have to either do a reload (and risk losing my terminal contents), or wait a minute or so for it to connect automatically. It seems that it is trying to connect to the WSL 2 VM but the connection attempt hangs for about half a minute before it times out (~20 seconds in this log, you can get this by subtracting 55 by 32 in the timestamps in this log). The extension should at least provide an option to change the timeout.
Here is my log:
[2021-09-06 17:45:32.534] [renderer3] [info] [remote-connection][ExtensionHost][840ad…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-09-06 17:45:32.537] [renderer3] [error] {"isTrusted":true}
[2021-09-06 17:45:32.540] [renderer3] [info] [remote-connection][ExtensionHost][840ad…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
[2021-09-06 17:45:32.563] [renderer3] [info] [remote-connection][ExtensionHost][840ad…][reconnect] resolving connection...
[2021-09-06 17:45:32.791] [renderer3] [info] [remote-connection][ExtensionHost][840ad…][reconnect] connecting to 172.17.148.156:35647...
[2021-09-06 17:45:32.793] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-09-06 17:45:32.794] [renderer3] [error] {"isTrusted":true}
[2021-09-06 17:45:32.795] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
[2021-09-06 17:45:32.797] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] resolving connection...
[2021-09-06 17:45:32.803] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] connecting to 172.17.148.156:35647...
[2021-09-06 17:45:55.469] [renderer3] [error] [remote-connection][Management ][10822…][reconnect][172.17.148.156:35647] socketFactory.connect() failed or timed out. Error:
[2021-09-06 17:45:55.470] [renderer3] [error] Error: WebSocket close with status code 1006
at WebSocket.<anonymous> (vscode-file://vscode-app/c:/Users/REDACTED/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:609:139641)
[2021-09-06 17:45:55.470] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] A temporarily not available error occurred while trying to reconnect, will try again...
[2021-09-06 17:45:55.470] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] waiting for 5 seconds before reconnecting...
[2021-09-06 17:45:55.544] [renderer3] [info] [remote-connection][ExtensionHost][840ad…][reconnect] reconnected!
[2021-09-06 17:46:00.473] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] resolving connection...
[2021-09-06 17:46:00.479] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] connecting to 172.17.148.156:35647...
[2021-09-06 17:46:00.537] [renderer3] [info] [remote-connection][Management ][10822…][reconnect] reconnected!
The remote WSL extension is a utterly pre-alpha-grade, inaccessible, and closed source mess right now. This issue is in the backlog so I don't expect it to be fixed any time soon. Go switch to Linux and don't use Windows!
@kevin-he-01 I'm sorry the experience is so rough right now around sleeping and WSL. While things around sleeping and reconnection have marginally improved over time, I agree with you that the experience here is not good.
From looking at your logs, I think the first issue is that while the client-side continues to attempt to reconnect after waking up from sleep, the server side will kill the remote extension host process after 3hrs. We use setTimeout
in the server implementation to drop disconnected clients after being disconnected for more than 3hrs. It looks like when putting the computer to sleep for more than 3hrs, when it wakes up, the event loop on the server process will execute the scheduled setTimeout
and the server will think that it has a client disconnected for more than 3hrs, and it will kill the remote extension host process. This would explain the Unknown reconnection token
error in the log. I will do two things in this area:
setTimeout
, which doesn't work on wall-clock time, but in program execution time, such that the server would drop a client only after really running for 3hrs, not after 3hrs elapsed in the real world.The last thing you are noticing where the first TCP/IP connection does not go through to WSL might stem from the WSL implementation at the OS level. Maybe immediately after waking up from sleep, their network adapters are not immediately set up correctly. I don't have an immediate idea on how to tackle this.
I wonder if it's worth mentioning that since upgrading my Windows laptop from an HP Elitebook 840 G3 to a Microsoft Surface Laptop 4, I have had very little trouble with this issue. Maybe because it's a much faster machine? i.e. are there timeouts on trying to reconnect that are too short for slower machines to complete on waking from hibernation?
@kevin-he-01 I've pushed improvements to the error message and to our use of setTimeout
that might get the server to drop a client when waking from sleep. These changes will be available in tomorrow's insiders (the build will run in about 16hrs), please give it a try and let us know if things are better after resuming from sleep.
@alexdima Thanks for your response. Just to clarify my first post here is about my old laptop without modern standby — An Asus Vivobook — and running WSL 1 (though the same issue occurred on WSL 2 on that machine as well). By running ps -ef
before and after reloading, I determined that you are right in that the remote server restarted itself leading to the Unknown reconnection token
error (The PIDs changed). Your setTimeout
fix will probably fix that.
My second post is about another reconnection issue on my new Surface Laptop 4. Like what drkvogel said, the experience is better with the new laptop (assuming I use WSL 1 on the new laptop), presumably because modern standby does not yank local (happening on localhost
) TCP connections:
I wonder if it's worth mentioning that since upgrading my Windows laptop from an HP Elitebook 840 G3 to a Microsoft Surface Laptop 4, I have had very little trouble with this issue. Maybe because it's a much faster machine? i.e. are there timeouts on trying to reconnect that are too short for slower machines to complete on waking from hibernation?
However, on my new laptop with WSL 2, the experience is not so good when I put my computer to sleep (modern standby). It is probably some Hyper-V/WSL2 network switching issue. It seems that the host-VM connection is tampered with when the computer enters some "deeper" sleep stage in modern standby (more than a few minutes). I get the "Disconnected. Attempting to reconnect" screen with no way to cancel and restart the connection attempt instantly (I can only reload or wait). I tried the hns
workaround but nothing changes.
@alexdima, I wonder if you can provide some insights on how VSCode remote communicates with WSL? Is there some heartbeat that detects disconnection? It seems that it is based on WebSocket just by looking at the logs. I know the extension is closed source. I am asking this because I need a way to reproduce the WSL 2 connection hang (~20 seconds reconnection time) using visible and simpler code snippets so I can report this issue to WSL and see what workarounds are available. Currently, I am not able to reproduce any network delay using any code I write: it only happens when VSCode tries to reconnect.
Apologies for having 2 issues to track at a time.
Today's insiders contains the setTimeout
fix, so the server side should now wait 3hrs for the client side to reconnect after a wake from sleep.
We have our own heartbeat mechanism to detect disconnection, but from looking at the log from your second post, it doesn't look like that is to blame here. In this case, it looks like we receive two "socket close events" from the network stack (we use WebSockets in an Electron renderer process, which use the Chromium network stack) and that is what triggers the reconnection.
But what is curious to me is the order of events. Here is your log a bit trimmed down:
[17:45:32.534] [ExtensionHost] received socket close event (wasClean: false, code: 1006).
[17:45:32.791] [ExtensionHost] connecting to 172.17.148.156:35647...
[17:45:32.793] [Management ] received socket close event (wasClean: false, code: 1006).
[17:45:32.803] [Management ] connecting to 172.17.148.156:35647...
[17:45:55.469] [Management ][172.17.148.156:35647] socketFactory.connect() failed or timed out. Error: WebSocket close with status code 1006
[17:45:55.470] [Management ] A temporarily not available error occurred while trying to reconnect, will try again...
[17:45:55.470] [Management ] waiting for 5 seconds before reconnecting...
[17:45:55.544] [ExtensionHost] reconnected!
[17:46:00.479] [Management ] connecting to 172.17.148.156:35647...
[17:46:00.537] [Management ] reconnected!
And the interpretation:
ExtensionHost
connection [A]Management
connection [B]Management
connection [C]To me, it is peculiar that connection [B], that is created immediately after connection [A], fails, while connection [A] succeeds. But in any case, it looks like it takes up to 20+ seconds for either WebSocket to succeed or fail. I don't really know how WSL is waking up from sleep and if this is to be expected or not.
To create a standalone repro, I guess you could try to reproduce by running a web server inside WSL and opening a web page in Edge/Chrome to it and from the web page creating 2 WebSockets to the WSL server. I think those two WebSockets will get closed (like they do for us) when waking the computer from sleep. Then, the page could try to create new WebSockets and log if that succeeds or not (or how long it takes for it to succeed). Maybe this also depends if the network conditions change for the laptop between going to sleep and waking up (e.g. closing the lid at work, opening it at home).
@aeschli Maybe we need to ask someone from WSL what is expected in this case.
This happens to me without the laptop going to sleep. Just pops up the dialog at random intervals. Stand-alone WSL sessions are unaffected. Doing the reload reconnects. It's starting to be unusable...
node sure is maxing out the CPU, but total memory usage is les than 50%.
I even increased the WSL memory to 15G, still happens after a few minutes:
[2021-11-01 11:13:54.707] [renderer1] [info] Starting extension host with pid 64896.
[2021-11-01 11:13:55.103] [renderer1] [error] [Extension Host] (node:64896) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Code --trace-deprecation ...` to show where the warning was created)
[2021-11-01 11:20:04.473] [renderer1] [info] [remote-connection][Management ][2ab16…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-11-01 11:20:04.473] [renderer1] [error] {"isTrusted":true}
[2021-11-01 11:20:04.474] [renderer1] [info] [remote-connection][Management ][2ab16…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
[2021-11-01 11:20:04.475] [renderer1] [info] [remote-connection][Management ][2ab16…][reconnect] resolving connection...
[2021-11-01 11:20:04.486] [renderer1] [info] [remote-connection][ExtensionHost][944e1…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-11-01 11:20:04.486] [renderer1] [error] {"isTrusted":true}
[2021-11-01 11:20:04.486] [renderer1] [info] [remote-connection][ExtensionHost][944e1…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
[2021-11-01 11:20:04.487] [renderer1] [info] [remote-connection][ExtensionHost][944e1…][reconnect] resolving connection...
[2021-11-01 11:20:05.561] [renderer1] [info] [remote-connection][Management ][2ab16…][reconnect] connecting to 172.27.206.17:33319...
[2021-11-01 11:20:05.562] [renderer1] [info] [remote-connection][ExtensionHost][944e1…][reconnect] connecting to 172.27.206.17:33319...
[2021-11-01 11:20:05.682] [renderer1] [info] [remote-connection][Management ][2ab16…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-11-01 11:20:05.682] [renderer1] [error] {"isTrusted":true}
[2021-11-01 11:20:05.683] [renderer1] [error] [remote-connection][Management ][2ab16…][reconnect][172.27.206.17:33319] received error control message when negotiating connection. Error:
[2021-11-01 11:20:05.683] [renderer1] [error] Error: Connection error: Unknown reconnection token (never seen)
at V (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:153441)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:144201
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at a.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:16680)
at c._receiveMessage (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:22005)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:21048
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at S.acceptChunk (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:14041)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:13389
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:139256
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at FileReader._fileReader.onload (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:137816)
[2021-11-01 11:20:05.683] [renderer1] [error] [remote-connection][Management ][2ab16…][reconnect] A permanent error occurred in the reconnecting loop! Will give up now! Error:
[2021-11-01 11:20:05.684] [renderer1] [error] Error: Connection error: Unknown reconnection token (never seen)
at V (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:153441)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:144201
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at a.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:16680)
at c._receiveMessage (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:22005)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:21048
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at S.acceptChunk (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:14041)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:85:13389
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:139256
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at FileReader._fileReader.onload (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:137816)
[2021-11-01 11:20:15.645] [renderer1] [error] [remote-connection][ExtensionHost][944e1…][reconnect][172.27.206.17:33319] the handshake timed out. Error:
[2021-11-01 11:20:15.646] [renderer1] [error] Error: Time limit reached
at o._createTimeoutError (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:141568)
at o._timeout (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:141514)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:141256
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at P.cancel (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:16086)
at D.cancel (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:16627)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:140974
at l.fire (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at P.cancel (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:16086)
at D.cancel (vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:69:16627)
at vscode-file://vscode-app/c:/Users/IVISLavender/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/workbench/workbench.desktop.main.js:610:140726
[2021-11-01 11:20:15.646] [renderer1] [info] [remote-connection][ExtensionHost][944e1…][reconnect] A network error occurred while trying to reconnect, will try again...
@rednevals Could you please also share the server logs?
Sorry, where can I find them? That was the VSCode log...
I have disabled all extensions, Docker and MiniKube. As soon as I open VSCode, it will do this after a few minutes of doing nothing.
Restarted the window:
[2021-11-01 12:10:23.823] [remoteagent] [info]
*
* Visual Studio Code Server
*
* Reminder: You may only use this software with Visual Studio family products,
* as described in the license https://aka.ms/vscode-remote/license
*
[2021-11-01 12:10:23.823] [remoteagent] [info] Extension host agent started.
[2021-11-01 12:10:24.060] [remoteagent] [info] [::ffff:172.27.192.1][731cc67b][ManagementConnection] New connection established.
[2021-11-01 12:10:24.118] [remoteagent] [info] [::ffff:172.27.192.1][eaa15342][ExtensionHostConnection] New connection established.
[2021-11-01 12:10:33.808] [remoteagent] [error] ptyHost was unable to resolve shell environment Unable to resolve your shell environment in a reasonable time. Please review your shell configuration.
[2021-11-01 12:10:33.808] [remoteagent] [error] ExtensionHostConnection#buildUserEnvironment resolving shell environment failed Unable to resolve your shell environment in a reasonable time. Please review your shell configuration.
[2021-11-01 12:10:33.816] [remoteagent] [info] [::ffff:172.27.192.1][eaa15342][ExtensionHostConnection] <10603> Launched Extension Host Process.
Dialog popped up after 10 minutes...
This is so strange. I have had this setup running for over two months without issue, building and deploying Docker containers locally. Just started acting up last week. The only other thing was a Windows update that I rolled back to see if it was the issue. It has not been reinstalled...
From the same timeframe:
[2021-11-01 12:10:34.528] [exthost] [info] extension host started
[2021-11-01 12:10:34.533] [exthost] [error] Error: EEXIST: file already exists, open '/home/sxl73/.vscode-server/data/User/workspaceStorage/9082b5237a2f4c358b35fe160505c54f/vscode.lock'
[2021-11-01 12:10:34.568] [exthost] [info] Lock '/home/sxl73/.vscode-server/data/User/workspaceStorage/9082b5237a2f4c358b35fe160505c54f/vscode.lock': Could not acquire lock, checking if the file is stale.
[2021-11-01 12:10:34.582] [exthost] [info] ExtensionService#_doActivateExtension vscode.extension-editing, startup: false, activationEvent: 'onLanguage:markdown'
[2021-11-01 12:10:34.582] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/extension-editing/dist/extensionEditingMain
[2021-11-01 12:10:34.608] [exthost] [info] ExtensionService#_doActivateExtension vscode.markdown-language-features, startup: false, activationEvent: 'onLanguage:markdown'
[2021-11-01 12:10:34.609] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/markdown-language-features/dist/extension
[2021-11-01 12:10:34.695] [exthost] [info] ExtensionService#_doActivateExtension hashicorp.terraform, startup: false, activationEvent: 'onLanguage:terraform'
[2021-11-01 12:10:34.695] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/extensions/hashicorp.terraform-2.16.0/out/extension
[2021-11-01 12:10:34.934] [exthost] [info] ExtensionService#_doActivateExtension vscode.debug-auto-launch, startup: true, activationEvent: '*'
[2021-11-01 12:10:34.934] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/debug-auto-launch/dist/extension
[2021-11-01 12:10:34.938] [exthost] [info] ExtensionService#_doActivateExtension vscode.git, startup: true, activationEvent: '*', root cause: vscode.github
[2021-11-01 12:10:34.938] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/git/dist/main
[2021-11-01 12:10:35.022] [exthost] [info] Lock '/home/sxl73/.vscode-server/data/User/workspaceStorage/9082b5237a2f4c358b35fe160505c54f/vscode.lock': The pid 148 appears to be gone.
[2021-11-01 12:10:35.022] [exthost] [info] Lock '/home/sxl73/.vscode-server/data/User/workspaceStorage/9082b5237a2f4c358b35fe160505c54f/vscode.lock': Deleting a stale lock.
[2021-11-01 12:10:35.023] [exthost] [info] Lock '/home/sxl73/.vscode-server/data/User/workspaceStorage/9082b5237a2f4c358b35fe160505c54f/vscode.lock': Lock acquired.
[2021-11-01 12:10:35.228] [exthost] [info] ExtensionService#_doActivateExtension vscode.github, startup: true, activationEvent: '*'
[2021-11-01 12:10:35.228] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/github/dist/extension.js
[2021-11-01 12:10:35.310] [exthost] [info] ExtensionService#_doActivateExtension ms-vscode.js-debug, startup: false, activationEvent: 'onCommand:extension.js-debug.clearAutoAttachVariables'
[2021-11-01 12:10:35.310] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/ms-vscode.js-debug/src/extension.js
[2021-11-01 12:10:39.952] [exthost] [info] ExtensionService#_doActivateExtension vscode.emmet, startup: false, activationEvent: 'onStartupFinished'
[2021-11-01 12:10:39.952] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/emmet/dist/node/emmetNodeMain
[2021-11-01 12:10:39.991] [exthost] [info] ExtensionService#_doActivateExtension vscode.merge-conflict, startup: false, activationEvent: 'onStartupFinished'
[2021-11-01 12:10:39.991] [exthost] [info] ExtensionService#loadCommonJSModule file:///home/sxl73/.vscode-server/bin/6cba118ac49a1b88332f312a8f67186f7f3c1643/extensions/merge-conflict/dist/mergeConflictMain
[2021-11-01 12:10:39.998] [exthost] [info] eager extensions activated
@rednevals Thank you for the extra logs. Here is what I can put together from the logs:
From the client log:
never seen
connection token, indicating that the server process inside WSL (or WSL itself) has been restartedFrom the server log:
~/.bashrc
, executing it takes longer than 10s. You should also be able to observe this when you try to launch a WSL terminal. This is something specific to your WSL machine. But that is OK, we detect that and proceed without the shell environment.All in all, I cannot say what exactly is going on. It looks like the server process inside WSL, together with the spawned extension hosts are crashing. It could also be that WSL itself crashes.
@rednevals I think there is also the WSL extension log that might provide more clues. @aeschli Do you collect extra logs somewhere that could be used to troubleshoot?
WSL seems to be running. I tried keeping a separate WSL shell open and it is not effected when this occurs.
I have code in my ~/.bashrc to ensure Docker and MiniKube are running. Recently I pulled the logic out into separate files that it now loads. I added a y/n prompt which does pause for the answer, but it is only ~1 second for me to answer the prompts:
For example:
# Is Docker daemon already running?
dockerPID=$(pgrep dockerd)
if [ $? -eq 0 ]; then
logMessage "Docker daemon already running, $dockerPID"
else
while true; do
read -p "Do you want to start Docker (y/n)?" yn
case $yn in
[Yy]* )
# Start Docker
sudo service docker start
logMessage "Waiting for Docker to start ..."
until pids=$(pidof dockerd); do
sleep 1
done
break;;
[Nn]* ) break;;
* ) echo "Please answer yes or no.";;
esac
done
fi
For now I have disabled sourcing these and all but the remote extensions. I'll give that a test, along with a separate WSL session open and report the results.
When is ~/.bashrc used, other than when I open a terminal session? Could these prompts be blocking things? The reason behind this code is that when Docker and MiniKube are running it taxes my laptop making the fan run quite a bit. I was trying to allow myself to work without these in situations where I don't need Docker or MiniKube. Maybe that was a mistake. Can you recommend another approach for this?
I was able to rework the Docker/MiniKube startup and some disk mounts in my profile scripts. This seems to have eliminated my issue. The read and sudo password prompts were getting in the way. There is no need to execute these every time I open a shell.
I moved the following from my ~/.bashrc to ~/.profile:
## Start Docker and MiniKube?
# Is Docker daemon already running?
dockerPID=$(pgrep dockerd)
if [ $? -eq 0 ]; then
logMessage "Docker daemon already running, $dockerPID"
else
nohup sudo -b dockerd < /dev/null > ~/work/logs/docker/dockerd.log 2>&1
until pids=$(pidof dockerd); do
sleep 1
done
fi
# Is MiniKube already running?
minikubePID=$(pgrep kube-apiserver)
if [ $? -eq 0 ]; then
logMessage "MiniKube already running, $minikubePID"
else
minikube start --vm-driver=docker --insecure-registry="$(hostname -I | cut -d ' ' -f1):5000"
fi
Some of this was helpful as well:
@rednevals We read the shell env by launching bash when we spawn the remote extension host. We would like to support that you setting an env variable in ~/.bashrc
is made available to extensions running on the remote extension host in WSL, so adding prompts there could definitely cause problems. But AFAIK we would continue even without those prompts being answered.
Can you please clarify, after tweaking your ~/.bashrc
to eliminate the sudo
and read
prompts, VS Code now works fine, no more disconnections?
So far, VSCode has not lost the connection since I implemented the changes...
Just wanted to chime in here and say that I've been experiencing this issue for quite some time. The error dialog does not give me an option to reconnect, only to reload. Upon reload, all terminal windows are reset (so it loses any running processes)
I WAS able to resolve it by creating a task in Windows task scheduler that reset the hardware clock:
Upon wakeup:
wsl.exe -u root sh -c hwclock -s
This worked up until a week or two ago. I am now experiencing the same issue despite resetting the clock. So, either a Windows Update or an update to wsl-remote broke this fix.
Here are the reconnection logs from vscode:
[2021-11-10 10:53:30.004] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] resolving connection...
[2021-11-10 10:53:31.626] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] connecting to 172.22.176.26:44655...
[2021-11-10 10:53:35.631] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] resolving connection...
[2021-11-10 10:53:35.640] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] connecting to 172.22.176.26:44655...
[2021-11-10 10:54:02.247] [renderer3] [error] [remote-connection][ExtensionHost][20432…][reconnect][172.22.176.26:44655] socketFactory.connect() failed or timed out. Error:
[2021-11-10 10:54:02.250] [renderer3] [error] Error: WebSocket close with status code 1006
at WebSocket.<anonymous> (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:39751)
[2021-11-10 10:54:02.285] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] A temporarily not available error occurred while trying to reconnect, will try again...
[2021-11-10 10:54:02.286] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] waiting for 10 seconds before reconnecting...
[2021-11-10 10:54:02.291] [renderer3] [error] [remote-connection][Management ][5fadd…][reconnect][172.22.176.26:44655] socketFactory.connect() failed or timed out. Error:
[2021-11-10 10:54:02.303] [renderer3] [error] Error: WebSocket close with status code 1006
at WebSocket.<anonymous> (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:39751)
[2021-11-10 10:54:02.304] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] A temporarily not available error occurred while trying to reconnect, will try again...
[2021-11-10 10:54:02.305] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] waiting for 10 seconds before reconnecting...
[2021-11-10 10:54:21.172] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] resolving connection...
[2021-11-10 10:54:21.172] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] connecting to 172.22.176.26:44655...
[2021-11-10 10:54:28.798] [renderer3] [info] [remote-connection][ExtensionHost][20432…][reconnect] reconnected!
[2021-11-10 11:04:35.017] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] resolving connection...
[2021-11-10 11:04:35.030] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] connecting to 172.22.176.26:44655...
[2021-11-10 11:04:35.088] [renderer3] [info] [remote-connection][Management ][5fadd…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
[2021-11-10 11:04:35.088] [renderer3] [error] {"isTrusted":true}
[2021-11-10 11:04:35.088] [renderer3] [error] [remote-connection][Management ][5fadd…][reconnect][172.22.176.26:44655] received error control message when negotiating connection. Error:
[2021-11-10 11:04:35.089] [renderer3] [error] Error: Connection error: Unknown reconnection token (seen before)
at V (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:54964)
at vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:45724
at l.fire (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at o.fire (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:85:16680)
at d._receiveMessage (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:85:22005)
at vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:85:21048
at l.fire (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at m.acceptChunk (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:85:14041)
at vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:85:13389
at vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:40536
at l.fire (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:69:1712)
at FileReader._fileReader.onload (vscode-file://vscode-app/c:/Program%20Files/Microsoft%20VS%20Code%20Insiders/resources/app/out/vs/workbench/workbench.desktop.main.js:610:39096)
[2021-11-10 11:04:35.089] [renderer3] [error] [remote-connection][Management ][5fadd…][reconnect] A permanent error occurred in the reconnecting loop! Will give up now! Error:
[2021-11-10 11:04:35.090] [renderer3] [error] Error: Connection error: Unknown reconnection token (seen before)```
Just an update. I'm on latest wsl-2 with 1.64.0-insiders and this problem persists. In the mornings, when I get on google-chrome recovers fine, but vscode completely freezes. This is before and after reconnecting VPN, which is slow for some reason as well.
Strangely, google-chrome seems completely unaffected running in wsl-2.
The only workaround I've found is to: wsl --shutdown
Unfortunately, this is a bit of a pain to do every day.
@mrjrieke Can you check the logs as .vscode-server-insiders/data/log
for anything interesting?
@aeschli
So, things seem to be better with the latest update of Windows 11 (Maybe KB4023057 and KB4023057)? I use code-insiders (1.64.0-insider) directly on wsl-2 on Windows 11 and things are pretty great!
Here's some details from the about screen on Code-insiders:
Version: 1.64.0-insider Commit: 4ccfe1100c3c5625804dbc20afd15ba311487883 Date: 2022-02-02T03:04:05.182Z OS: Linux x64 5.10.60.1-microsoft-standard-WSL2
I'm waiting for an official build of VS Code to have all the latest code that is showing in code insiders before I move back to using a stable VS code.
I also experience loss of access to WSL often, and the only way I've figured out to get it back is to restart my computer.
How do I close VS Code, when it has lost connection to WSL without losing data?
I'm still experiencing this.
WSL 2
VSCode Version: 1.87.1 (user setup) Commit: 1e790d77f81672c49be070e04474901747115651 Date: 2024-03-06T00:21:36.258Z Electron: 27.3.2 ElectronBuildId: 26836302 Chromium: 118.0.5993.159 Node.js: 18.17.1 V8: 11.8.172.18-electron.0 OS: Windows_NT x64 10.0.22631
This may be a more general connectivity issue as I’ve noticed other services failing after standby. Only way to fix it I’ve found is wsl shutdown/restart
On Sat, Mar 9, 2024 at 7:56 AM Andrew Nelson @.***> wrote:
I'm still experiencing this.
WSL 2
VSCode Version: 1.87.1 (user setup) Commit: 1e790d77f81672c49be070e04474901747115651 Date: 2024-03-06T00:21:36.258Z Electron: 27.3.2 ElectronBuildId: 26836302 Chromium: 118.0.5993.159 Node.js: 18.17.1 V8: 11.8.172.18-electron.0 OS: Windows_NT x64 10.0.22631
— Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-remote-release/issues/3454#issuecomment-1986896240, or unsubscribe https://github.com/notifications/unsubscribe-auth/AE4UKP7NUFUTX7KCLGNXX6DYXMWJVAVCNFSM4PP6FXMKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJYGY4DSNRSGQYA . You are receiving this because you were mentioned.Message ID: @.***>
Steps to Reproduce:
In the morning, VS Code says it's lost its connection with WSL and needs to reload the window. However, calling
wsl -l -v
shows that the WSL 2 VM is indeed still running.I've written about Modern Standby specifically because I don't know if this behavior occurs with normal standby as well, but I imagine it's probably specific to Modern Standby because Windows is still active there, but not during normal standby afaik.