Integrated terminal renders client-side input with latency

[caer]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS/Web Information

• Web Browser: Firefox 126.0.1 (64-bit)
• Local OS: Windows 11 Pro
• Remote OS: Rocky Linux 9.3
• Remote Architecture: ARM64
• code-server --version: 4.89.1 effc6e95b4ad1c5ac5f9083ec06663ba4a2e005c with Code 1.89.1
• Network Topology: US East (Client) <-> Cloudflare Tunnel <-> EU Helsinki (Host)

Steps to Reproduce

1. Open code-server in Firefox
2. Open the integrated terminal
3. Type some text

Expected

Near-instant rendering of entered text on the client-side.

Actual

Entered text is displayed with lag/latency, seemingly equivalent to the RTT to the remote host.

Logs

No response

Screenshot/Video

No response

Does this bug reproduce in native VS Code?

I did not test native VS Code

Does this bug reproduce in GitHub Codespaces?

I did not test GitHub Codespaces

Are you accessing code-server over a secure context?

• [X] I am using a secure context.

Notes

I have a round-trip latency of ~120ms from my client to the host. However, I was hoping that--at least for text entry--I wouldn't observe input lag, since I don't observe any input lag when using the integrated editor windows.

The only related issue I found was https://github.com/coder/code-server/issues/163, but I wanted to double-check if anything has changed since then, since the issue didn't appear to be actually resolved.

[code-asher]

Yeah the editor text is static; VS Code just has a local copy of the text, you can edit it locally, and on save it sends the whole thing and writes it to the disk.

The terminal process runs on the remote and is dynamic; a key press could do anything. VS Code has to wait to hear back from the remote to know how the terminal state has been changed.

But, there is a setting that will output the character you type immediately in a lighter color, then once it hears back from the terminal it resolves the differences if any. It is called "local echo" and there are a few settings to tweak it. I think it is already on by default, but you could try setting the latency threshold to 0. Some programs are ignored by default, like vim, tmux, and some others (also configurable). A warning though, some folks have seen visual glitches with local echo on.

[caer]

Ahhh got it, architecturally...that make sense. 😭

I just finished playing with the Local Echo feature you mentioned. For others' reference, these are the settings I used in settings.json:

{
    "terminal.integrated.localEchoEnabled": "on",
    "terminal.integrated.localEchoStyle": "bold",
    "terminal.integrated.localEchoLatencyThreshold": 10,
}

Some notes:

• terminal.integrated.localEchoEnabled defaults to auto (enabling it for remote workspaces), so changing it to on for code-server is kind of redundant...but I did it just to be sure.
• terminal.integrated.localEchoStyle seems to take either a hex color code, or one of: 'bold', 'dim', 'italic', 'underlined', 'inverted'.
• terminal.integrated.localEchoLatencyThreshold when set to 0 would force echo always, but given your warning--and weird things I've read elsewhere--I set it to 10, which is about the limit of what I can perceive latency-wise.

With these settings, the input lag is still annoyingly perceptible, but it feels a little bit better; thank you for the tip @code-asher!

[code-asher]

Glad it helps at least a little! I will go ahead and close the issue since I think we are stuck with the way things are now.

Maybe there are clever ways to improve the local echo or the latency, but I think that work would be best done upstream in VS Code rather than a patch here anyway.

As an aside, have you ever used ssh over mosh to your remote machine? Just curious how it performs in comparison to VS Code's terminal in your setup, if you have. I wonder if VS Code can take any lessons from mosh.

[caer]

I was considering mosh! For this deployment, though, I want to rely entirely on the integrated terminal so that I can manage all access and ingress through the browser (via Cloudflare gateways), and avoid exposing any ports (including SSH/UDP) outside that workflow.

...that said, for the purpose of this issue, I just configured a deployment with mosh. Details:

• My intended network topology is US East (Client) <-> Cloudflare Tunnel <-> EU Helsinki (Host).
• However, I was too lazy to configure out-of-browser tunneling for mosh, so I reconfigured my topology (for this test) to be US East (Clientl <-> EU Helsinki (Host) with ports 22/TCP (for the ssh / mosh client) and 60069/UDP (for the mosh-server) opened up for direct connection
• There is probably some overhead that the Cloudflare tunnel adds, but it is likely not much based on the latencies I see when manually pinging the remote data center from my location.

With this setup, mosh was much much more enjoyable than the integrated VS code terminal; I almost would have believed I was using a local terminal. It would be really wonderful if VS code--or some kind of wild extension to it--could leverage or take inspiration from mosh.

Hope this info is useful; thank you for all the help! ❤️

[code-asher]

Thank you for going out of your way to test that!! Very interesting result.

Since Mosh is UDP and WebSockets are TCP I wonder how much that affects things versus the Mosh protocol itself probably just being better. Maybe things would be better if VS Code was reworked to use WebRTC, at least for the terminal.

An extension is an interesting idea! I suppose it would have to be done as a web extension so it runs client-side: implement a JavaScript client for Mosh unless one already exists, run over WebRTC to the Mosh server... I wish I had the bandwidth to experiment, maybe one day. Even just starting with a browesr-based Mosh client outside of VS Code would be fairly interesting.