High Input Lag on 3D games

[Dogway]

PCSX2 version: pcsx2-v1.5.0-dev-3138-gfbafd4420-windows-x86

PCSX2 options:

Software or Hardware mode indistinctly. Using Safe Preset

Plugins used: Default: GSdx32-AVX2 Spu2-X LilyPad 0.12.1 FWNull Driver 0.7.0

Description of the issue: There's a high game input lag in 3D based games, rarely below 6 frames of lag. Here are some tests done with frame advance using the minimum recorded frames for fastest event paths.

MGS3 Subsistance (6 frames) GTA Vice City (6 frames) GTA San Andreas (10 frames, 166ms!) Sengoku Basara X (6 frames) Tokyo Xtreme Racer 3 (7 frames) Taiko Drum Master (6 frames) Silent Scope (6 frames) Time Crisis 3 (5 frames) Capcom Fighting Evolution (4 frames)

Generally it's 6 frames of game input lag which is one of the highest recorded on current emulation software compared to Dolphin (5), Citra (4), Retroarch (4-5) and possibly Cemu and RPCS3 (they feel snappy)

How to reproduce the issue:

-Enable "Recording Tools" -Load a game -Press the frame advance hotkey -Press a fast event triggering button on your pad -Press again the frame advance hotkey as many times (counting them) until the event starts the animation

PC specifications: Operating System: Windows 7 x64 SP1 CPU: Intel i5-4670K @ 4.1Ghz GPU: Nvidia GeForce GTX 1070

[Jacoby1218]

I’ve noticed that too, what controller are you using?

[Dogway]

I use a 360 wireless controller but this is unrelated because I'm only measuring the emulation path input lag through the frame advance procedure. I guess judging by the picture you might already know but here is a good review of controller input lag.

[KrossX]

Is there data on PS2 + CRT input lag? It would be nice to have some, specially for GTA-SA. Might be tricky though as you could need a light in the controller for when a button is pressed and when pressure is greater than deadzone.

[tadanokojin]

Honestly I have a lot of problems with these numbers. Not the least of which is I can't independently verify any of them.

[KrossX]

You need a high speed camera and a controller that has a light on it.

3rd Strike 4.7 Dodonpachi DaiOuJou (White) 3

Dodonpachi seems like a good test point.

[tadanokojin]

I mean even barring a comparison to a control (which they don't provide), they aren't explicitly telling me where these numbers come from.

Here are some tests done with frame advance using the minimum recorded frames for fastest event paths.

What is "fastest event path" and how is it determined to be the fastest?

In DC2 I get numbers ranging from 4 to move the start menu cursor to 10 frames to swing a sword. A lot of which could probably be explained by animation blending and easing or natural delay introduced by the developers.

In Raw Danger! It's about 11 frames to move the cursor in the start menu, but I happen to know that game runs at 20fps internally.

Dolphin (5), Citra (4), Retroarch (4-5) and possibly Cemu and RPCS3 (they feel snappy)

Even if I granted this comparison made sense (I don't think it does). Where are these numbers even coming from?

[QuaiGoner]

In Flatout games input lag almost unplayable (fullspeed)

[Dogway]

What is "fastest event path" and how is it determined to be the fastest?

Do an array of tests in a game and take "the fastest" of the events. As you said in your DC2 example between the 4 and 10 frame lag take the fastest/shorter, 4. When you test enough games and events you start to paint a clear picture. I also think an input lag test with a real PS2 on a CRT should be made to clearly define the lag introduced through emulation, mind you the PS2 controller will likely introduce half a frame of input lag. Unfortunately I don't own a 60fps camera.

Even if I granted this comparison made sense (I don't think it does). Where are these numbers even coming from?

To be as unbiased as possible all tests were performed as explained on the issue description, with TAS tools using frame advance. Cemu and RPCS3 lack these tools. While I also think PS2 2D games have at least 1 frame of game/emulation input lag, on 3D games it's specially jarring. When you add 6 frames to an extra frame for LCD and another one for the controller (at least), we are talking about 8 frames (133ms) minimum of input lag.

Edit: Found an input lag test with a PS2 3D game on a CRT. (3.86 frames of lag, accounting controller)

[tadanokojin]

As you said in your DC2 example between the 4 and 10 frame lag take the fastest/shorter, 4.

Yeah but there are thousands of things to test. I'm asking you for the specific things that gave you the number. We need to be able to reproduce it.

I also think an input lag test with a real PS2 on a CRT should be made to clearly define the lag introduced through emulation

I agree. That information would be useful.

To be as unbiased as possible all tests were performed as explained on the issue description, with TAS tools using frame advance. Cemu and RPCS3 lack these tools.

What games? What did you test? Where can I read the results? What's your sample size for the other emulators? Do we even need this comparison?

You're worried about bias but your sample size for PCSX2 is 9 games which I don't think is representative.

[Dogway]

I'm asking you for the specific things that gave you the number. We need to be able to reproduce it.

I test the fastest in gameplay, either jump, punch, turn (if digital input), etc. Eventually I also test Start to check it aligns to the measured minimums. These aspects are not usually given in tests because it's asummed you measured enough events and times, but if you need a specific event to reproduce I will post them later on.

I agree. That information would be useful.

I linked a video on my edit to a test with a real PS2, I didn't find what the display type was but in case it was an LCD I assume it to be one with zero lag. For reference Mega Man X8 (a 3D game) has an input lag of 64.459 (3.867 frames)

What games? What did you test? ...

An average of an array of games similar to the one presented here. I test enough games until I find a pattern, obviously I'm not going to test the whole library but it's wise to pick games developed with the PS2 on mind, suggest games and I will try to test them. I did a study on input lag for 3D systems in retroarch that you can find on the link on top. As you can see they vary from 3 to 5 frames.

Now that we have a few comparison samples of real measurings on real hardware (Mega Man X8, Mega Man X7, Dodonpachi DaiOuJou (White)...), maybe we could run specific comparisons on PCSX2, but as I said, the events used for these measurings were not redacted so it depends on if you find them valid tests and/or valid games.

[sandman332]

Can confirm this, makes many first person shooter games unplayable.

[tadanokojin]

@Dogway

but if you need a specific event to reproduce I will post them later on.

I'd like to know what you are testing, yes, so others can test it an eliminate any external factors. You can also post some block dumps so others can test.

In the meantime you can try disabling MTVU if it isn't already and seeing if that reduces latency any.

An average of an array of games similar to the one presented here.

Honestly just lose the comparison, you clearly can't or won't give me any details about it and frankly I don't think it would tell us much anyway.

Feel free to test other games in PCSX2, though. Jak series might be interesting.

maybe we could run specific comparisons on PCSX2

I agree. A good comparison against real hw would give us a delta and help us understand how much of the lag is emulation overhead.

[willkuer]

Wouldn’t it make more sense to check something the in-game button press memory address and check how many frames it takes to register the button press event?

Also wouldn’t you expect that the emulation overhead is game-independent as long as it’s not bottlenecked by the user’s system?

[Dogway]

Did more tests, MTVu has no effect, neither backend except for Yakuza. I start invoking the events from the first playable scene in the game, then I test several events including Start and moving through the Start options to rule out intrinsic animation delays.

Testing through frame advance in Windowed Mode (OpenGl SW and HW, hyphen - means random not backend). If you need the profiles, let me know.

5     (Jump) (SW/HW) Jak and Daxter - The Precursor Legacy (USA)

4-5   (Jump) (SW/HW) Mega Man X8 (PAL)

4     (Jump) (SW/HW) God of War (PAL)

4     (Jump) (HW)    God of War II (PAL)

6     (Punch/Square)    (HW) Yakuza (USA)
6     (Start Menu Move) (HW) Yakuza (USA)
7     (Move/Start)      (HW) Yakuza (USA)
11    (Punch/Square)    (SW) Yakuza (USA)
11    (Start Options)   (SW) Yakuza (USA)
12-13 (Move/Start)      (SW) Yakuza (USA)

6     (Start Options)     (SW/HW) Gran Turismo 4 (USA)
5     (Brake/Camera View) (SW/HW) Gran Turismo 4 (USA)

7-8   (Jump)        (SW/HW) Kingdom Hearts (PAL)
6     (Start)       (SW/HW) Kingdom Hearts (PAL)
5     (Start Items) (SW/HW) Kingdom Hearts (PAL)

9-10  (Start/Start Options/Jump) (SW/HW) Ico (PAL)
15    (Move)                     (SW/HW) Ico (PAL)

4     (Start)     (HW) Monster Hunter (PAL)
4     (Move/Turn) (HW) Monster Hunter (PAL)
5     (Start)     (SW) Monster Hunter (PAL)
4-5   (Move/Turn) (SW) Monster Hunter (PAL)

5     (Start)        (SW/HW) Tekken 5 (USA)
6     (Cross/Square) (SW/HW) Tekken 5 (USA)

4     (Start)              (SW/HW) Grand Theft Auto III (PAL)
5     (Car Brake -Square-) (SW/HW) Grand Theft Auto III (PAL)
6     (Jump -Square-)      (SW/HW) Grand Theft Auto III (PAL)

10    (Start/Start Options/Move Turn/Car Brake) (SW/HW) Grand Theft Auto - San Andreas (PAL)
14    (Jump)                                    (SW/HW) Grand Theft Auto - San Andreas (PAL)

10-11 (Shoot/Crouch)        (SW/HW) Black (PAL)
12-13 (Start/Start Options) (SW/HW) Black (PAL)

[water111]

The input latency is confusing. I measured Jak 1's input latency with a light on a controller and a high speed camera.

On PS2, I would expect around 5 frames:

• 0 : button pushed
• 1 : IOP reads controller (once per frame)
• 2 : EE receives IOP DMA with button press and calls ja-push
• 3 : animation system runs, puts animation frame in next frame's DMA list
• 4 : VU processes DMA list and writes to frame buffer
• 5 : frame with animation is visible

With the camera, I measured between 5 and 6 frames of latency (usually 6) on a PS2 with a CRT TV.

On PCSX2, I would expect another frame to read the controller, and probably a few more on graphics.
But I got between 11 and 18 frames of latency. (11,13,14,12,13,17,13,18,11,15)

I am not sure where the huge latency is coming from, but I did find two interesting things:

1). In PCSX2, some game frames are just not shown. Even if the game is internally running at 60 fps and PCSX2 is running full speed, it will just skip frames at random. I found an animation which is 17 frames, and plays as frame 1,2,3,....,17 on a real PS2. In PCSX2, some frames a just skipped, so you saw 1,1,3,4,5,6,6,8,9,9,11,12,13,13,13,16,17. Watch this video and see how it is choppy in the second half https://www.youtube.com/watch?v=IMjHCIwb7hc, which matches the high speed video of the screen. The game is running at 60 fps internally here, or the numbers at the top would turn red.

2). The EE/VU timing is very off, so the games may sometimes be running at a different frame rate on PCSX2 and PS2. Jak mostly runs at the right frame rate, but in certain areas, it internally runs at 30 fps on PCSX2 where PS2 runs at 60 fps. I think this is because some EE code takes around 2x more cycles than it should. See the game profiler here when the game goes at 30 fps in PCSX2: https://imgur.com/a/WwBHpY8 . The upper bar is EE, and the yellow/white sections are an EE-based renderer. This fits within 100% on PS2, but is over 120% on PCSX2. The lower bar is VU1, which has around 2x higher utilization on PS2.

[tadanokojin]

I found an animation which is 17 frames, and plays as frame 1,2,3,....,17 on a real PS2. In PCSX2, some frames a just skipped, so you saw 1,1,3,4,5,6,6,8,9,9,11,12,13,13,13,16,17.

Interesting. Just spitballing but I wonder if the alt vsync PR addresses this any,

[tadanokojin]

I ran some tests from @Dogway's results. I ran these tests by buffering the input for 2 frames. I'm including that buffering in my results. All tests are NTSC-U.

In GTA III I tested the pause menu. I found it was 4 or 5 frames before the background appears. I did note however that the audio cue happens on the 3rd frame. https://youtu.be/_KJMuUMM5T8

In GT4 I got about 6 frames for the audio cue and the menu begins to appear on the 7th. Exiting it I'm noticing it's only 3 frames for the audio cue. https://youtu.be/FC2Y6aQakjw

In Ico, which was the one I was most interested to test because of the high number. I tested it by holding the analog stick in the direction opposite where the character was facing and waiting for the character to move. The result I got was 8 frames. Which is a little more than half what Dogway is reporting. Not sure if this is due to differences in regions or if Dogway tested a different move animation. https://youtu.be/hW15-DHc_a8

I got 10 frames for the start menu to appear. https://youtu.be/myE6cU5hZ1E

[sandman332]

So, are there any band-aid fixes or anything that can be done from the users end to somewhat fix this issue? Makes all the games I have played almost unplayable and some definetly are.

[mirh]

Well, for starters, mods could have not deleted my comment referencing the stuttering problems of the nvidia driver #2307. Considering we have yet to solve that hot potato, I'm not sure we really can take at face values results from geforces (unless under linux)

I wonder what best case input lag of the emulator could be then? Say, with unlocked framerate and in the bios menu.

[ramapcsx2]

The upper bar is EE, and the yellow/white sections are an EE-based renderer. This fits within 100% on PS2, but is over 120% on PCSX2. The lower bar is VU1, which has around 2x higher utilization on PS2.

@water111 Could you share how you got into that performance counter debug mode?
It's probably using hardware that PCSX2 doesn't fully / correctly emulate, so I wouldn't trust it too much, but I'd like to try this with modified VRender / VBlank timing.
Maybe it makes a difference.

[water111]

These performance bars should work correctly for EE - they are just based on the timers which PCSX2 does correctly. Other performance menus aren't emulated correctly and display all zeros, I think because PCSX2 doesn't support the EE performance counter registers. The VU bar is a little weird because of how PCSX2 handles VU1 timings.

Here's a link to a patch that you can apply to PCSX2 to get into this mode. It's a bit of hack because it needs to emulate a dev kit with extra RAM and patch several memory values at specific times.

https://gist.github.com/water111/e4a0af85968daad1ac478573ae07b48c

(I think it also changes the vsync timing, so maybe you want to remove this)

If you boot Jak and daxter NTSC/PAL/Japan version, it should start up in the debug mode. Then press L3 to open the menu, then click Display, then profile.

I am fairly certain this isn't a vsync timing issue, but instead an EE timing issue. Most of the time is spent in a renderer where almost every instruction can be dual issued.

[ramapcsx2]

That's insanely clever xD Thanks!

[MrCK1]

@Dogway I'm assuming this is still an issue for you or has anything changed in the recent dev builds?

[sandman332]

Something tells me this is still a pretty big issue.

[mirh]

People on reddit were still reporting this (even though ofc "everybody has it when it comes to complaining, but nobody when it's time to understand")

[Dogway]

As a headnote, I'm waiting to have a 1.6.0 RC to test, the one I downloaded wasn't built with recording tools enabled.

[tadanokojin]

They'll be disabled for the release due to some issues with it. You'll have to stick with the dev builds for now.

[Dogway]

last dev build has it also disabled...

My bad: copied over the wrong file

[tadanokojin]

Just checked dev 3391 and it's enabled.

[Dogway]

A little update with the most affected titles (dev 3391):

9-10  (Start/Start Options/Jump) (SW/HW) Ico (PAL)
7     (Turn over/Lang Sel Menu)  (SW/HW) Ico (PAL)

10-11 (Shoot/Crouch)        (SW/HW) Black (PAL)
12-13 (Start/Start Options) (SW/HW) Black (PAL)

[mirh]

Thank you for sticking with us, and even reporting precise numbers. Did we already rule out this could have something to do with the problems with nvidia cards and stuttering? Can you try to put EnableVsyncWindowFlag=enabled in pcsx2_ui.ini, set aspect ratio to "Fit To Window/Screen" and play in fullscreen?

[Dogway]

This is a little different issue although I would also like to see stutter improve. I'm frameskipping after an input event in pause mode so it's the internal lag of the system/emulator what is measured, I will test GTA as well but things don't look promising. Since the minimum recorded is 4 frames (GOW) it tells there's room for improvement for most other between 5 and 6 frames (leaving out input, monitor and buffer/vsync related lag), maybe dissecting the most offending ones can reach to some conclusions (Black, GTA SA).

[ABU3BGE]

so is there a fix ?

[phantomolecule]

I've never had this problem until very recently . Strangely enough it happens on dev build 3391 as well as an older dev 3306 build (I thought it might've been because I updated). Other emulators and games work just as expected, and the delay is present even in the "test device" section of LilyPad, not just in games, so it doesn't seem to have anything to do with graphics cards or performance. Hopefully a fix is in the works as there seems to be many people affected, and the delay is significant enough that I just won't use pcsx2 for the time being EDIT: thought maybe it was my DS3 drivers so I tested it with keyboard and mouse, no change, same input delay.

[KrossX]

LilyPad has a test device thing? You could try the other APIs and see if there's a difference. Other than that, there's the xpad and pokopom plugins that could be checked with xinput controllers.

[ABU3BGE]

i try all plugin nothing changed

[phantomolecule]

Yeah if you right click your input device under "device diagnostics" there's an option to "test device", it brings up a pretty simple list of all the inputs and shows when the value changes. Thanks for the suggestion, just tried the other plugins in 3391, still no change unfortunately.

[ABU3BGE]

i think i fixed it or at least lower the delay tell the dev to see what the problem was the fix is simple uncheck automatic gamefix you can find it under system . if anyone know what is used in automatic gamesfix or can i find the ini file for the automatic fix not the manual fix tell me . i tested it with shinobido it kind of fixed it when you try it tell me if it worked and what was the game have fun

[phantomolecule]

Interesting, I'm glad you're able to enjoy your game more than you could before. That it only lessened the delay makes me think it might have not fixed the real underlying issue. My issue might be a little different, as I see the delay even when testing the gamepad in PCSX2 outside any games running, using "test device" under "device diagnostics". In fact, the keyboard responds the same way as the controller when tested, in or out of game, but only inside PCSX2. Maybe I should open up a new issue? As I'm realizing this topic is specifically for 3D games.

[KrossX]

Well, ingame lag would still be relevant I think. You could open up a new issue about lag on the lilypad test window. It could be a bug with how the input is polled and presented or intentional to avoid the test window from being flickery.

[phantomolecule]

I can't see how it would be an issue specifically with the lilypad test window as the issue is still present using other controller plugins, as well as consistently across all games I've tried (various games I've had no problems playing in the past). The test window was just how I isolated performance and specific games as a factor. Downloading a fresh copy of pcsx2 and confirming nvidia control panel vsync is off still hasn't made it budge, very strange.

[KrossX]

LilyPad Diagnostic Window Timer

The plugin creates its own window, it does not go through the emulator loop. The link is for the diagnostics window that has its own message handling. You can see it sets a 200ms timer to update the information. Thus, this was intentional, possibly to avoid a flickering dialog window. It's a window just to test if the device is working, not intended to be a latency benchmark.

[phantomolecule]

Very interesting, thanks for clearing that up. All the more bizarre that the delay is seemingly identical to the one I find in-game, although 200ms seems a bit high. Admittedly my experience with input delay is mostly in music production, but the gap seems to be around 100ms or so, enough to make a difficult racing or action game unplayable. ATM I'm messing with the framelimiter to see if I can increase or decrease the delay based on framerate, seeing as how I can't rule out graphics card or performance with a separate diagnostic. No change so far unfortunately.

[tadanokojin]

Try setting in PCSX2_vm.ini, think it's called VSyncQueueSize or something.

[mirh]

SO, AS I WAS SUSPECTING: After my fifth or sixth outburst on reddit, a very collaborative user stood up and helped me with some tests (all of this was on an intel gpu, on w10, with on capcom fighting jam). And once we could manage to get some shape or form of exclusive fullscreen, every input lag problem was solved, and "comparable to real PS2" (or 1 frame behind at most every now and then, possibly just being introduced by the display) The real hardship was getting there in the first place though.

It turns out for some reason "put EnableVsyncWindowFlag=enabled in pcsx2_ui.ini" wasn't working for him. Thankfully he knew about W10's fullscreen optimizations that convert every dx11 borderless fullscreen window to fullscreen. And as for opengl, were they don't work, we ended up just forcing the famous WS_POPUP flag with winspy++. (and of course you still need fit to screen)

Please, anybody test that.

[tadanokojin]

And how was this measured?

[mirh]

With a 60fps camera.

[water111]

Here is some data. This is unrelated to the fullscreen stuff, but is related to games feeling laggy/unplayable.

I ran jak 1 and set my capslock button to be jump. I filmed with a 120 fps phone camera, and measured from caps lock LED on to jumping animation start. With the latest PCSX2, it took 30 video frames (15 PS2 frames) for the animation to start. I ran it a few more times and got 15, 17, 19, 15. This is similar to the numbers I got before.

Then I ran the same thing on the PS2. Note that my TV caught fire recently so I was using a component to VGA box that probably has some lag, and a crappy VGA monitor which probably also has some lag. From finger on button to jump, it took 18/20 video frames (9 to 10 PS2 frames). I suspect that there are a few frames of latency in the converter from how it feels, but I don't have hard data. Note that I was measuring from when my finger touched the button, not when the button was pushed, because it was too hard to judge when the push actually happens.

I noticed that PCSX2 also was dropping a lot of frames for me, generally the UI was stuttering, and there was a freeze of the UI for around 3 seconds on startup. I checked if the UI was getting blocked on some OS thing with ETW and it wasn't - it was at 100% CPU usage during the freezes. I ran it in the VS debugger and noticed it happens at roughly the same time as when there are "access violations" caused by the JIT.

I dug a little deeper and found that there's a UI event generated every time the EE recompiler exits recompiled code. See SysCoreThread.cpp, in DoCpuExecute(). The Cpu->Execute() function pointer will point to recExecute in iR5900-32.cpp when the EE recompiler is running, which returns on every page fault (happens whenever new code is encountered and needs to be compiled).

The UI event is UI_EnableSysActions, which is kind of slow because it does some stuff related to the save slots and their names. It is really silly to put this UI stuff in the CPU execute / recompiler loop, as it can exit a huge number of times when a game first runs, or if the game requires a lot of recompiling.

If you comment out the call to UI_EnableSysActions() inside of SysCoreThread::DoCpuExecute, the game becomes much less choppy. Also, the latency decreases from 15 frames to 11 frames. I don't understand PCSX2's UI design, but everything seems to work correctly with this commented out.

Adding this change makes PCSX2 playable for me, and otherwise it is very unpleasant. If anybody is more familiar with the PCSX2 UI, it would be great to get an explanation of how this UI_EnableSysActions function is supposed to work, and I can work on a better fix than just commenting it out.

[pixelherodev]

I dug a little deeper and found that there's a UI event generated every time the EE recompiler exits recompiled code. See SysCoreThread.cpp, in DoCpuExecute(). The Cpu->Execute() function pointer will point to recExecute in iR5900-32.cpp when the EE recompiler is running, which returns on every page fault (happens whenever new code is encountered and needs to be compiled).

Ohhh, that's brilliant! Page faulting on unknown code so it can be compiled on demand!

... not entirely on-topic, I know; sorry, been obsessively playing with dynarecs lately.

[ramapcsx2]

Yeah, this is highly unusual code, but it is there for a good reason. (I don't claim to understand any of it, mind). So it's possible that there was much less GUI code to check initially, but it grew in complexity over time. In that case, a redesign of the slow parts would probably be a good idea. Of course this requires someone who fully understands all the implications to do it :)

But yeah, the GUI hooks do serve an important function, as weird as they seem to be.