Determine how best to measure perceived input latency (character input to screen)

[vadimkantorov]

@DHowett-MSFT> We should figure out how best to measure input perf. This may require #4001.

original content

I'm trying out the new release of WT. Previously I've used wsltty to ssh into a server.

Immediately after switching to WT it feels that characters appear on screen a bit slower than in wsltty. Not sure how to measure / debug / confirm properly.

[vadimkantorov]

When I step on a very long line in vim and just press key "Right", WT glitches noticeably, sometimes even hanging for what looks like ~100ms. I don't know if it's cursor/character rendering that's slow or underlying controls being slow or sth else...

This may sometimes be the consequence of slow SSH / internet, but in this time it's not the case I think. wsltty renders cursor / characters a bit faster

[DHowett-MSFT]

So, the software I usually use to measure things like this is called "typometer". It's an old piece of Java software that injects input and waits for the output to be rendered on the screen.

We've gotten a number of improvements in rendering speed since 0.11 landed, so I'm inclined to say that your issues are going to be largely addressed.

If you have VS 2019, I welcome you to try out a Release build from master :smile:

[vadimkantorov]

So, the software I usually use to measure things like this is called "typometer". It's an old piece of Java software that injects input and waits for the output to be rendered on the screen.

Spooky! :) Looks like it's a valid "testing infra" feature request? (installing Java for end-user input on this can be quite a heavy dependency) :) Cursor rendering speed should also be supported for measurement

[vadimkantorov]

If you have VS 2019, I welcome you to try out a Release build from master 😄

Good news it's improved, but it's too heavy-lifting for me for now. I wish there were *.msixbundle artefacts in some nightly builds?

[DHowett-MSFT]

I'll repurpose this one into the Task for "determine a good way to measure performance". :smile:

[DHowett-MSFT]

This will require #4001.

[vadimkantorov]

a related request (also by me) on time-to-rendering measurements and loading: https://github.com/microsoft/terminal/issues/6409

[lhecker]

As a stopgap you can use an app called "Is It Snappy?" with a modern iPhone (those supporting Slow-Mo videos). At 240 FPS recording speed you can get decently accurate end-to-end latency measurements that way.

[NelsonMinar]

Some 2024 measurements with method explained: https://chadaustin.me/2024/02/windows-terminal-latency/

[lhecker]

I have the same monitor as you do, running at 144 Hz, but a better CPU (AMD 5950X) and GPU (RTX 4090), so my experience may be better because of faster hardware. Additionally, I don't have WSL 1 nor Windows 10 available at the moment, which may also result in quite a different experience. Lastly, I've customized my keyboard to have a poll rate of 1000 Hz.

I noticed that your best and worst results in the first table are pretty much exactly 30 and 60 FPS respectively, which is, if I'm entirely frank, a little bit suspicious. I would expect there to be a lot more spread, the results to be generally lower, and for them to not coincide with such common dividers.

For instance, on Windows 11 build 26060, conhost, WSL2 and bash, I consistently get an input latency of 16ms which is 2 frames. For Windows Terminal I get a latency of 25-32ms (3-4 frames).

Just to be sure, could you check whether your input latency is better when you disable G-SYNC for your monitor?

[NelsonMinar]

could you

I am not the person who wrote the blog post I linked so I can't answer your questions, sorry.

[zadjii-msft]

@chadaustin might be able to answer those questions though - or provide any other thoughts he may have here.

I'd reckon the Terminal's input latency is from the conpty buffering:

https://github.com/microsoft/terminal/blob/5f3a857192b2fa6f4e5ae87eb3ee59a1571515a2/src/renderer/base/RenderEngineBase.cpp#L72-L78

[chadaustin]

Is It Snappy? is pretty handy for ad hoc measurements, but people like rtings.com that need automated tests over time use dedicated hardware to synthesize keypresses and detect pixel changes: https://www.rtings.com/company/input-lag-tool

In practice, since you have no control over USB input and the monitor's internal latency, you could probably detect pixels changing within the OS.

I haven't looked deeply beyond my blog post linked above, but I'd love to know why the conpty-based terminals are all in a tier slower than MinTTY/WSLTTY.

[lhecker]

@chadaustin Oh I didn't want to ping you simply out of my misunderstanding in the collapsed ("Resolved") comment above, but now that you're here...

I've since realized that I recently made a latency reduction in this PR: #15608 That one released in Windows Terminal v1.19.2682.0. Could you try and update to the latest version and see if that fixes your issue? 1.19 contains additional changes that should reduce latency.

I have very similar hardware and the same monitor (running at 144 Hz) and I'm getting an input latency of 16ms for conhost (Windows 11 build 26100, which contains the same latency improvements) and 25-32ms for Windows Terminal.

[chadaustin]

Ah, I passed over the resolved comments in my first reply.

Wow, nice work. I just gathered five measurements on my Windows 10 machine with Windows Terminal 1.19.10821.0.

Same methodology: banging a pencil into my QMK macro pad, terminal near the top of the screen, and measured the delta in frames between the 240 Hz frame AFTER my pencil makes contact with the key, and the pixels starting to change:

62.5, 33.3, 33.3, 25.0, 54.2

33.3 ms median is a huge improvement over the previous 66.7 median ms.

Thank you!

[chadaustin]

I added an update to the top of my blog post that links to this issue: https://chadaustin.me/2024/02/windows-terminal-latency/