Investigate and improve input latency

[pdjstone]

The mouse latency in particular could be a lot better. Need to investigate why the lag occurs - whether it's due to the Emscripten/SDL or how Arculator is using SDL. Arculator running on native SDL has much better latency.

My understanding of how the main loop works when running in the browser wrt mouse:

• In emscripten_main.c, arcloop is run once every browser "frame" via emscripten_set_main_loop. This is driven from JavaScript using requestAnimationFrame which is tied to your monitor refresh rate and usually runs at 60fps (i.e. once every 16ms).
• arcloop calculates the number of milliseconds since the last frame and calls arc_run(ms) to emulate that number of milliseconds of emulator time.
• In main.c arc_run calls mouse_poll_host and keyboard_poll_host which should get the latest mouse event data from the browser. It then calls execarm to run the actual emulation.
• In emulator time, keyboard_poll is set to run once every 10000us (10ms). It calls get_mickeys to get the latest mouse movement data into the emulator.

In theory, we should be seeing 1 or 2 get_mickeys calls per frame because each browser frame is 16ms and get_mickeys is being called every 10 emulated ms. There should be new mouse event data every frame (i.e. before every call to arcloop) But we need to verify this.

One change I've already made from upstream Arculator is to change the order of calls in arc_run from:

execarm();
joystick_poll_host();
mouse_poll_host();
keyboard_poll_host();

to:

joystick_poll_host();
mouse_poll_host();
keyboard_poll_host();
execarm();

This should mean that mouse events get processed before the next frame is rendered, improving latency by one frame.

[pdjstone]

I added a flag to archimedes-live which should help with debugging and measuring latency. If you add #dbglatency to the URL, it will add some listeners to the mouse up/down/move events when the mouse is captured. The background will flash red when a mouse button is clicked and will flash blue when the mouse is moved. With a similar program running inside the emulator you can measure the delay between the browser getting mouse events and when the emulator does.

Both Chrome and Firefox have pretty good profilers built in which also record each graphics frame. Here's the Firefox recording - https://share.firefox.dev/3Xs1UL3. If you move over the Screenshots bar, you can see that the emulator background generally changes a single frame after browser background. This seems good but I've not really dug into the profile any further than that. Chrome seems to be about the same.

[pdjstone]

I tried reverting the above change - putting the execarm() call before the input polling and it does indeed make the latency worse by one frame (https://share.firefox.dev/3CGMYkc). Good to confirm that the change is an improvement.

[pdjstone]

I added some logging to print out the timestamps (performance.now()) of when a mousedown event is received in different parts of the code. I also logged the VIDC background colour before and after arc_run. This lets us see what's happening when the background goes red when the latency test program is running. I get the following logs:

JS mousedown 37252
emscripten_main mousedown before arc_run: 37255 vidc_bg: 00000000 
keyboard_poll mousedown: 37255 
emscripten_main mousedown after arc_run: 37261  vidc_bg: 000000ff

This shows that the mouse event is received by keyboard_poll in the same loop iteration that SDL receives it. It also shows that the VIDC background color changes to red in the same iteration. There are only 9ms between the first and last log line above. This suggests that the 1-frame latency is in the rendering code, not in the input pipeline.

[matthewbloch]

Finally sat down to measure with Is It Snappy on the iPhone which records at 240fps and lets me mark input & output, assuming you film the keyboard & screen. So in theory, it can measure at a resolution of 4.16ms.

So my simple test was to hit F12 and see how long til the command line * pops up, nice and obvious as it makes the whole screen jump. It's a bit less obvious to decide on which frame the input happened; I was just looking to see when my finger bounced off the bottom of the keypress. But still...

Under RISC OS 3.11 on the emscripten build on Firefox, I measured 191ms and 170ms on successive tests.

The same build on Windows, same source tree, I measured 125ms and 120ms.

For reference, on the same computer (Windows 11 laptop with an nVidia 2060) I loaded up indie banger Bezier because I know it's all hand-coded. I tried to measure the latency of moving between two menu items and mayyybe it was 29ms but I think I would need to improve my button-down detection because I was struggling to pin down the input frame and it could have been less. My monitor latency is quoted as 9ms.

So - 120ms on native seems like it has a lot of room for improvement, and Emscripten adds a (perfectly believable) >50% tax to that. I'd be interested to find some numbers for other emulators, and of course on native - if you have an iPhone and an A3000 plugged in to test :)

[matthewbloch]

On that basis I'd be interested in looking over the C codebase first to see why it's as sluggish as it is, because the emulation does seem to run at full speed.

But also I wonder whether it's possible to short-cut the SDL / Emscripten screen update loop and narrow the gap between native & web builds.

[pdjstone]

I spent a few evenings with Is It Snappy, testing a bunch of different things on different hardware, OSes and browsers. I put my results into a spreadsheet, and I'll try to summarise the trends:

• Your latency numbers seem way too high! Most things ran with < 100ms latency even on slow-ish hardware. Are you using a wireless mouse or keyboard perhaps? Was your native Windows build of Arculator running fully native or on WSL2?
• For me, sub-100ms latency is usable, sub-60ms feels snappy.
• WebGL (which SDL on Emscripten uses by default) definitely has better latency than Javascript/Canvas2D especially on Chrome.
• On Windows, Chrome has better latency than Firefox.
• On one of my Linux machines, Chrome seems to have much worse latency than Firefox. On my other machine they're about the same.
• Chrome on Windows has much better latency in fullscreen, shaving off 10-20ms compared to non-fullscreen. I suspect it opts into some fullscreen low-latency API that Windows makes available for games.
• Archimedes Live on Chrome on Windows outperforms the official Arculator build! (However the official build doesn't yet have my tweak that improves latency by one frame)
• jsbeeb has about the same latency as Archimedes Live. However because we have mouse, the latency is more noticeable.

I'm hoping we can still improve arculator-wasm latency slightly (i.e. by one frame / 16ms) however it seems that much of the latency is platform/browser/hardware/phase of the moon dependent!

[matthewbloch]

Weird! OK I mean that's encouraging. There is a bit of judgment in marking the input frame, but I'll try again with your method and add some rows to the spreadsheet. No wireless. I will try to replicate your measuremenets and using my (2020 Windows, Ryzen, nVidia) laptop alone.

[pdjstone]

On Chrome we may be able to use the pointerrawupdate event to get lower latency mouse input (see https://navidz.github.io/pointerrawupdate.html)