Code server disconnects after browser inactivity for a while

AlkaSaliss commented 1 month ago

Is there an existing issue for this?

[X] I have searched the existing issues

OS/Web Information

Web Browser: Chrome, Firefox
Local OS: macOS, Windows
Remote OS: Amazon linux 2 (Sagemaker)
Remote Architecture: x86_64 GNU/Linux
code-server --version: code-server: v4.16.1, Code: 1.80.2

Steps to Reproduce

launch code server on Sagemaker notebook instances
Connect from local machine using Chrome, or Firefox
When the go computer is idle (e.g. in screen saver mode) for a while (the duration is not consistent, but 15min should do), or when the browser is out of focus for a long time, the vscode client page gets disconnected: the file system is no longer responsive (we have a blue clock icon and a horizontal line that scrolls continuously, see screeshot); and the terminal no longer respond also
Refreshing the browser page brings the connection back to normal

Expected

The vscode code client should not be disconnected when the computer has access to internet

Actual

Vscode client page gets disconneted when computer is idle or when browser is out of focus for a while

Logs

No response

Screenshot/Video

vscode-disconnection

Does this bug reproduce in native VS Code?

I did not test native VS Code

Does this bug reproduce in GitHub Codespaces?

I did not test GitHub Codespaces

Are you accessing code-server over a secure context?

[ ] I am using a secure context.

Notes

No response

code-asher commented 1 month ago

Weird, I did not think VS Code had any kind of idle timeout mechanism. I will try to leave code-server open and see if I can reproduce, but in the meantime can you share the browser console logs?

AlkaSaliss commented 1 month ago

Weird, I did not think VS Code had any kind of idle timeout mechanism. I will try to leave code-server open and see if I can reproduce, but in the meantime can you share the browser console logs?

Hi, thanks for the quick reply I've managed to reproduce and copy the following chrome logs. For additional context, I'm using code-server on AWS Sagemaker notebook instances, and exposing it via jupyter server proxy with configuration similar to:

c.ServerProxy.servers = {
    'codeserver': {
        'launcher_entry': {
            'enabled': True,
            'title': 'Code Server',
            'icon_path': '/home/ec2-user/SageMaker/.code-server/icon.svg'
        },
        'command': ['/home/ec2-user/SageMaker/.code-server/bin/code-server', '--auth', 'none', '--disable-telemetry', '--bind-addr', '127.0.0.1:{port}', '--open', '/home/ec2-user/SageMaker/'],
        'environment': {
            'XDG_DATA_HOME': '/home/ec2-user/SageMaker/.xdg/data',
            'XDG_CONFIG_HOME': '/home/ec2-user/SageMaker/.xdg/config',
            'SHELL': '/usr/bin/bash'
        },
        'absolute_url': False,
        'timeout': 30
    }
}

The disconnection issue reproduce consistently with other users using it on other sagemaker instances.

Here are the chrome console logs:

log.ts:421  INFO Resolving connection token (my-sagemaker-id.notebook.eu-west-1.sagemaker.aws)...
log.ts:421  INFO Resolved connection token (my-sagemaker-id.notebook.eu-west-1.sagemaker.aws) after 1 ms
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96)...
content.js:1 Uncaught Error: Could not establish connection. Receiving end does not exist.
    at b (content.js:1:322352)
b @ content.js:1
Show 1 more frame
Show less
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96) was successful after 122 ms.
/codeserver/stable-2ccd690cbff1569e4a83d7c43d45101f817401dc/static/out/vsda_bg.wasm:1 

       Failed to load resource: the server responded with a status of 404 (Not Found)
/codeserver/stable-2ccd690cbff1569e4a83d7c43d45101f817401dc/static/out/vsda.js:1 

       Failed to load resource: the server responded with a status of 404 (Not Found)
codeserver/:1 Refused to execute script from 'https://my-sagemaker-id.notebook.eu-west-1.sagemaker.aws/codeserver/stable-2ccd690cbff1569e4a83d7c43d45101f817401dc/static/out/vsda.js' because its MIME type ('text/plain') is not executable, and strict MIME type checking is enabled.
webWorkerExtensionHost.ts:115 The web worker extension host is started in a same-origin iframe!
u @ webWorkerExtensionHost.ts:115
v @ webWorkerExtensionHost.ts:131
start @ webWorkerExtensionHost.ts:124
Hr @ extensionHostManager.ts:139
j @ instantiationService.ts:119
createInstance @ instantiationService.ts:85
f @ extensionHostManager.ts:65
tb @ abstractExtensionService.ts:741
sb @ abstractExtensionService.ts:726
rb @ abstractExtensionService.ts:713
gb @ abstractExtensionService.ts:413
(anonymous) @ extensionService.ts:105
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a)...
log.ts:421  INFO [Service Worker] registered
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a) was successful after 153 ms.
webWorkerExtensionHostIframe.html:1 An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing.
log.ts:421  INFO Updating additional builtin extensions cache
codeserver/:1 Uncaught (in promise) Error: A listener indicated an asynchronous response by returning true, but the message channel closed before a response was received
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a)...
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96)...
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a) was successful after 219 ms.
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] reconnected!
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96) was successful after 367 ms.
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] reconnected!
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a)...
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96)...
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-6fe7071b-1ae2-46f0-847c-d58496780b2a) was successful after 166 ms.
log.ts:421  INFO Creating a socket (renderer-Management-a02104d9-979f-4baa-a958-7c9c1a4f4e96) was successful after 287 ms.
log.ts:421  INFO [remote-connection][ExtensionHost][6fe70…][reconnect] reconnected!
log.ts:421  INFO [remote-connection][Management   ][a0210…][reconnect] reconnected!

code-asher commented 1 month ago

Strange...does running it with --log trace give you anything more? I doubt it, but might be worth a shot.

I left code-server open yesterday but could not reproduce, maybe it has something specific to do with AWS. :thinking: I tried running it again now with the same flags you are using (well different port and path) but so far seems stable.

I usually see 1006 when there is a reverse proxy that is not configured to proxy web sockets but I am not sure if that is the case here since it does eventually connect. Still, we might need to look at logs on the AWS side to figure it out.

code-asher commented 1 month ago

Actually, one other thing to check, in NGINX for example there is a default timeout of 60 seconds if a web socket has no activity. Maybe AWS has a similar thing? Except, I thought VS Code sent pings to prevent this. Maybe we can check the web socket activity in the browser console to make sure they are being sent.

AlkaSaliss commented 1 month ago

Strange...does running it with --log trace give you anything more? I doubt it, but might be worth a shot.

I haven't seen the --log trace parameter (maybe due to old version I'm using), however I've ran it using the --verbose flag and here's the Chrome logs I've retrieved:

log.ts:401 TRACE ChatService#onWillSaveState: Persisting 0 sessions
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 2 chars
log.ts:401 TRACE Layout#layout, height: 961, width: 1790
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 0 sessions
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 2 chars
log.ts:421  INFO [remote-connection][ExtensionHost][99b57…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][ExtensionHost][99b57…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][ExtensionHost][99b57…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][ExtensionHost][99b57…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 1/6. invoking socketFactory.connect().
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-99b57fd3-e9f2-489a-93d2-16fdad177eda)...
log.ts:421  INFO [remote-connection][Management   ][43c8f…][reconnect] received socket close event (wasClean: false, code: 1006, reason: ).
log.ts:441   ERR CloseEvent
log.ts:421  INFO [remote-connection][Management   ][43c8f…][reconnect] starting reconnecting loop. You can get more information with the trace log level.
log.ts:421  INFO [remote-connection][Management   ][43c8f…][reconnect] resolving connection...
log.ts:421  INFO [remote-connection][Management   ][43c8f…][reconnect] connecting to WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)...
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 1/6. invoking socketFactory.connect().
log.ts:421  INFO Creating a socket (renderer-Management-43c8f7e7-e31d-4b39-aa04-ebc0f47a49f9)...
log.ts:421  INFO Creating a socket (renderer-ExtensionHost-99b57fd3-e9f2-489a-93d2-16fdad177eda) was successful after 171 ms.
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 2/6. socketFactory.connect() was successful.
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 3/6. sending AuthRequest control message.
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 4/6. received SignRequest control message.
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 5/6. sending ConnectionTypeRequest control message.
log.ts:401 TRACE [remote-connection][ExtensionHost][99b57…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 6/6. handshake finished, connection is up and running after 248 ms!
log.ts:421  INFO [remote-connection][ExtensionHost][99b57…][reconnect] reconnected!
log.ts:421  INFO Creating a socket (renderer-Management-43c8f7e7-e31d-4b39-aa04-ebc0f47a49f9) was successful after 336 ms.
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 2/6. socketFactory.connect() was successful.
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 3/6. sending AuthRequest control message.
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 4/6. received SignRequest control message.
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 5/6. sending ConnectionTypeRequest control message.
log.ts:401 TRACE [remote-connection][Management   ][43c8f…][reconnect][WebSocket(my-sagemaker-id.notebook.eu-west-1.sagemaker.aws:443)] 6/6. handshake finished, connection is up and running after 405 ms!
log.ts:421  INFO [remote-connection][Management   ][43c8f…][reconnect] reconnected!
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 0 sessions
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 2 chars
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 0 sessions
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 2 chars
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 0 sessions
log.ts:401 TRACE ChatService#onWillSaveState: Persisting 2 chars
log.ts:401 TRACE Layout#layout, height: 961, width: 1099

I usually see 1006 when there is a reverse proxy that is not configured to proxy web sockets but I am not sure if that is the case here since it does eventually connect. Still, we might need to look at logs on the AWS side to figure it out.

Any idea where I can get code-server local logs on my AWS instance ? I can't tell which logs are relevant to this issue but here's codeserver's data folder structure if that can help:

Actually, one other thing to check, in NGINX for example there is a default timeout of 60 seconds if a web socket has no activity. Maybe AWS has a similar thing? Except, I thought VS Code sent pings to prevent this. Maybe we can check the web socket activity in the browser console to make sure they are being sent.

I'm quite unfamiliar with web technologies, any hint on what to check exactly ?

Thanks

code-asher commented 1 month ago

Sorry for the delayed response.

I've ran it using the --verbose flag

Perfect, this is equivalent to --log trace.

Any idea where I can get code-server local logs on my AWS instance

That is the right spot for code-server's logs, specifically the coder-logs directory and the logs directory. I find it difficult to navigate the logs directory so usually I delete them all and then launch code-server again, that way there is only one directory. 😆

There might be additional logs in AWS, but I am not familiar enough with AWS to know where they are. Probably in the AWS dashboard somewhere.

I'm quite unfamiliar with web technologies, any hint on what to check exactly

To check AWS's web socket timeout, I am not sure unfortunately.

To check the web socket messages:

Open the developer console (Shift+F12 should be one way to open it, not sure if that works on macOS though)
Refresh the page
Go to Network > WS to filter by web sockets
Click on one of the sockets
Click Messages

You should see at least one message every five seconds or so.

ujjawal-khare commented 1 month ago

I am facing the same issue. Please let me know if you guys need some logs from me or find any solution for this.

coder / code-server