beatmania IIDX 16 - unusual delay between backing track and key audio

[lain-2p]

PCSX2 version: v1.5.0-dev-3135

• Notes: VSync toggling did not seem to remedy the issue, nor did changing the refresh rate in pcsx2_vm.ini to 59.82 as if it were potentially time-based from NTSC-I video modes like the earlier games.

Adjusted various EE/VU settings to no avail, and none of the speed hacks seem to impact it either.

Internal resolution/deinterlacing method did not seem to impact the problem, nor did fullscreen/windowed or using OpenGL (however OpenGL HW seems to suffer from a seperate frame pacing issue every 10 seconds or so, which is why I am opting to use DX11)

The in-game display offset function does not make up for this issue, as it equally offsets the already out of sync key and backing audio.

Tried multiple settings in SPU2-X, also to no avail.

PCSX2 options: Preset 2 (safe), speedhacks enabled, VSync on, aspect 4:3

Plugins used: GSdx 1.1.0 SSE4 DirectX11 HW, bob tff deinterlace everything else default

Description of the issue: Once in a song, there is a set backing track, on top of which the key samples play to form the whole song. Currently in the emulator, this backing track is roughly one beat off from where it should be, as if it started playing ever so slightly late. Unlike the issue with the non interlaced IIDX games, the speed at which the tracks play or the delay between the two doesn't seem to change.

This may be hard to demonstrate to someone not familiar with the series, as the issue is somewhat subtle if you aren't able to play along to the beat, but if needed I can try to demonstrate the problem in a video.

How to reproduce the issue: Boot the game up, select your controller and default name, go in to FREE mode, and play Quasar in the Alphabetical folder Q. Try to line up your input to the backing track independent of the note crossing over the red line. if replicated, it will show as POOR or GOOD (and generally be audibly wrong). The input for the first key in Quasar (1) is the square face button (quasar was chosen as the first few notes are simple and distinct)

Last known version to work: Others have seemingly had more success in the past, but I don't know the details of their PCSX2 setups. https://youtu.be/sPCEMPIC2xE for example

PC specifications: Intel i5 4670k @4.2GHz, AMD Radeon R9 290, 12GB 1600MHz DDR3 RAM, SSD, Windows 10

[kevhensen]

Hi, As per my understanding, you experience a lag between the visual and the sound of your game. Is that correct?

If so, can you take a screen capture of your sound settings (SPU2-X Settings window) by going to Config->Sound (SPU2)?

You might want to try all the different synchronizing modes and other sound settings to see if it amplifies, reduce or fix the problem and give us insight about what happens.

Here are 2 configurations to reduce lag to a minimum you could try as an example:

Hope it helps,

Kevin

[Prism019]

PCSX2 Version: aa03c83126225f92ec98569646b0d2aeffc45360

This issue should affect all beatmania IIDX games starting from 9th style onward (I've tested 6th style (not present) and 12 HAPPY SKY (present)). The issue also seems to be present in at least one GuitarFreaks/DrumMania game (I tested GFDM V2 (present)).

PCSX2 Options: Aspect Ratio: 16:9 All other settings default.

Plugin settings: Everything default.

APU Settings: Interpolation: 0 - Nearest Disable Effects Processing: enabled Module: PortAudio PortAudio API: 2 - JACK Synchronizing Mode: None (Audio can skip.) Latency: 3ms (I also tried it with Synchronizing Mode set to TimeStretch with 15ms latency. Still present. Also tried it with Module set to SDL with the SDL API set to JACK, but that just crackled a lot.)

PC Specification: AMD FX-8350 @ 3.6GHz, AMD Radeon R9 280X, 16GB 1333MHz DDR3 RAM, 7200 RPM HDD, Arch Linux JACK server set to 48000Hz, 256 buffer size (5.3ms latency)

[refractionpcsx2]

Is this still an issue on latest master?

[tjbautista01]

Maybe late but just want to chime in on this issue.

Tried out with PCSX2 1.6.0 and I don't think it's an SPU issue. Although setting the framerate to ~59.83 for 3rd-8th style fixed the eventual early drifting issue around halfway into the track as discussed in a separate issue. Autoplaying tracks confirms that.

For this one though, it seems more like an issue related to the GPU/PAD (?) as the game registers the inputs very late like some good chunk of milliseconds between the keypress and when the game in the emu registers a hit, to the point that I have to hit quite early just to get a Flashing Great/Just Great in the game. Probably leaning more towards GPU as how early I would have to hit before the notes hit the red line will depend on the HI-SPEED settings in the game (very early for higher speeds, less early for slower speeds).

Using HIDDEN+ on versions with the feature and hitting just above the lane cover can help with replicating this. For speeds around 2.5, set the lane cover up to just below where the judgements are (~200ish white number?)

It happens using either the keyboard or a IIDX controller passed thru a PS-USB converter, in which I don't think input latency is an issue as I can get Perfects just fine in DDR with a PS1 dance pad passed thru the converter.

[refractionpcsx2]

have you tried the latest development build for this using the normal framerates? we know 1.6.0 is wrong, but we changed the framerates recently.

[tjbautista01]

have you tried the latest development build for this using the normal framerates? we know 1.6.0 is wrong, but we changed the framerates recently.

Tried the latest dev build (v1.7.0-dev-842-g95e7045db) and it seemed to have fixed the issue with the older IIDX style without me even having to manually input the refresh rate so kudos to that.

With regards to gameplay timing though, it's really leaning towards being a GPU issue than a PAD issue, though it might be a combination of both. Like said previously, I had to press very early to get a Just Great and how early varied with the HISPEED settings. This was with IIDX 6th style.

To confirm, I loaded up IIDX GOLD and set the Timing Adjustment to -5 and lo and behold, I could time my inputs right. This will work with versions DistorteD and above. For IIDX HAPPY SKY, I set the adjustment to -3 as this version was particularly notorious for having very late timing.

To make sure if PAD wasn't lagging, I loaded up DDR Supernova2 as given DDR's timing was more msec-based than frame-based like IIDX so any display lag shouldn't affect it much. Confirmed also with DDR Extreme that yeah inputs were on time.

[refractionpcsx2]

Okay there's a couple of things you can try

Set your output module in the SPU2 settings to Portaudio, click configure under it and make sure the latency is the lowest it can go. Also in your PCSX2_vm.ini in the inis folder there is an option for VsyncQueueSize, set that to 0.

after that, see what the timing is like without adjustment.

[tjbautista01]

Okay there's a couple of things you can try

Set your output module in the SPU2 settings to Portaudio, click configure under it and make sure the latency is the lowest it can go. Also in your PCSX2_vm.ini in the inis folder there is an option for VsyncQueueSize, set that to 0.

after that, see what the timing is like without adjustment.

Tried it. Still have to press early.

For reference, here is how early illustrated by the HIDDEN+ lane cover. The red line is where I'm supposed to press once the notes are close to it but instead I have to press as the notes approach the top of the lane cover. (Ignore the POOR I took it mid-game)

[refractionpcsx2]

Are you not supposed to press it when it reaches that line just above where it says poor? 🤔 I didn't realise it was so low...

okay can you try this build too? https://github.com/PCSX2/pcsx2/files/5767008/pcsx2-inputlag2.zip

[tjbautista01]

Are you not supposed to press it when it reaches that line just above where it says poor? 🤔 I didn't realise it was so low...

okay can you try this build too? https://github.com/PCSX2/pcsx2/files/5767008/pcsx2-inputlag2.zip

That line just above POOR is a measure indicator aka which measure are you in the track now. If you play 100% minimooG in HAPPY SKY those measure indicators are gone as said track has no known time signature.

That aside i'll try that build

[tjbautista01]

Are you not supposed to press it when it reaches that line just above where it says poor? 🤔 I didn't realise it was so low...

okay can you try this build too? https://github.com/PCSX2/pcsx2/files/5767008/pcsx2-inputlag2.zip

Update: Tried the build. Timing definitely improved. I can do -3 on GOLD and -1 on HAPPY SKY. for 6th Style I still have to press somewhat early but it's more on par with a PS2 plugged into a laggy HDTV using composite/AV out albeit a value higher (regarding GOLD and HS in which I use -2 and +1 respectively).

Tried DDR as well and timing was also better.

So it really was a combination of input lag and GPU lag but at least one's covered now.

[Prism019]

I wish I could test, but my computer isn't able to be set up right now, and I don't trust my laptop to be able to keep up with pcsx2. When my computer is set up, I will definitely be able to test changes on linux.

[refractionpcsx2]

Are you not supposed to press it when it reaches that line just above where it says poor? 🤔 I didn't realise it was so low... okay can you try this build too? https://github.com/PCSX2/pcsx2/files/5767008/pcsx2-inputlag2.zip

Update: Tried the build. Timing definitely improved. I can do -3 on GOLD and -1 on HAPPY SKY. for 6th Style I still have to press somewhat early but it's more on par with a PS2 plugged into a laggy HDTV using composite/AV out albeit a value higher (regarding GOLD and HS in which I use -2 and +1 respectively).

Tried DDR as well and timing was also better.

So it really was a combination of input lag and GPU lag but at least one's covered now.

Interesting, i mean that build is basically VsyncQueueSize = 0, but there is a small change to how often the pad updates to, though i don't suspect we can get it any better than what is in that build.

[refractionpcsx2]

Okay one last try, can you try again with this build and included plugins found here: https://github.com/PCSX2/pcsx2/issues/3008#issuecomment-756307822

please follow the instructions on that post to get the benefit.

[tjbautista01]

Okay one last try, can you try again with this build and included plugins found here: #3008 (comment)

please follow the instructions on that post to get the benefit.

Done as instructed. Same results as earlier with the inputlag2 one.

[refractionpcsx2]

Okay thanks for testing :)

[refractionpcsx2]

Out of curiosity, do you need to adjust it all on a PS2 to achieve correct timing?

[tjbautista01]

Out of curiosity, do you need to adjust it all on a PS2 to achieve correct timing?

when my ps2 is connected to an hdtv, yes for HAPPY SKY and above (all games below HS don't have the adjust option so i'm the one adjusting)

On my current setup, no. Current setup is PS2 outputting component video to a component to HDMI converter and then connected to the HDMI port of a ViewSonic VA2359-SMH monitor. When playing IIDX the monitor's set to Game mode and response time to Ultra Fast and IIDX plays as if I was playing on a CRT. Although the response time thing might not be so relevant as I can still get good results on Standard response time but eh better safe than sorry.

[refractionpcsx2]

I'm just trying to establish how different it is, that's all, although I don't know if it can be improved much more and the guys who seem to be sensitive to input lag seem to be pretty happy with some of the builds I've posted related to it. I just don't want to be chasing unicorns if it needs adjusting on the actual console too.

[tjbautista01]

I'm just trying to establish how different it is, that's all, although I don't know if it can be improved much more and the guys who seem to be sensitive to input lag seem to be pretty happy with some of the builds I've posted related to it. I just don't want to be chasing unicorns if it needs adjusting on the actual console too.

So I just tested it again with VSync disabled and the aspect ratio set to fit to screen (in this case, 16:9) and the timing of input got better than last time. It's now at the level you'd expect plugging the PS2 into an HDTV via composite aka better than using the PS3.

[refractionpcsx2]

yeah well generally I would suggest using VSync off and the framelimiter enabled and let that do the sync, that generally has better timing as your monitors vsync rate is unlikely to fit nicely with the games running speed. But that's good to hear!

[tjbautista01]

yeah well generally I would suggest using VSync off and the framelimiter enabled and let that do the sync, that generally has better timing as your monitors vsync rate is unlikely to fit nicely with the games running speed. But that's good to hear!

Yeah it seems that with versions lower than Happy Sky, players would just have to adjust themselves to compensate (Thanks Konami for using frame-based timing)

[refractionpcsx2]

I guess so ;)

[limyz]

@tjbautista01 I will test IIDX sometime, I have several misfires despite having the correct input on GuitarFreaks. @refractionpcsx2 As the bug is no longer related to the SPU, this issue should be closed and transferred to #3008. For music video games with audio sensitivity, you should ideally use PortAudio with Windows WASAPI to achieve minimum latency.

[tjbautista01]

Am using PortAudio with lowest latency settings and vsync disabled (input lag is worse with it on) but yeah input lag still remains, even when using a keyboard.

[tjbautista01]

Tested with latest nightly build and recommended settings. Still has the "need to hit real early" issue like playing on a laggy HDTV. Either it's because of the engine (frame-based timing and makes use of display's refresh rate rather than internal clock like most rhythm games), input lag (present even if playing on a keyboard, but this seems to have been fixed in previous nightlys?), or maybe something else entirely.

[limyz]

I know this sounds dumb, have you tried setting your display to 59Hz? Also, input lag is subjective and unavoidable. I need to test on IIDX - somehow this was magically solved in the forums (https://forums.pcsx2.net/Thread-Input-lag-on-PS2-Music-games).

[tjbautista01]

Not possible. Monitor only can do 60Hz. Would it matter though as vsync is disabled? This is weird as pop'n music, which is similar to IIDX except it uses msec-based timing, is on time (press notes to line, get Perfect/Just Great) while in IIDX you have to press way above like halfway between the bottom of where the combo counter is and the top of the hit zone.

[avih]

Not possible. Monitor only can do 60Hz. Would it matter though as vsync is disabled?

It's possible. Most Displays (TVs and monitors) support both 60 (PC) and 59.94 Hz (NTSC), the latter is usually chosen when you set the display rate to 59 (because the dialog only supports integer values, 59 is a known "code" for 59.94).

So the windows compositor can still introduce lag of its own, which will be smaller if the display is at 59.94.

[tjbautista01]

59Hz not in selectable modes

[limyz]

Use CRU (https://www.monitortests.com/forum/Thread-Custom-Resolution-Utility-CRU) to add 59.94Hz.

[avih]

Also, depending on driver, in fullscreen it might switch to exclusive mode, which will bypass the windows compositor and remove the lag it adds.

However, there will always be some lag. The goal here is to minimize it - not to eliminate it.

[tjbautista01]

Connected to a display that can do 59Hz (which is also where my PS2 is connected when playing console IIDX) and even with response time set to Ultra Fast the latency persists, even on 9th style which bumped the resolution from 240p to 480i

[limyz]

The Acer KA220HQ is a TN monitor and has a rather lacklustre response time of 5ms. Your best bet is to use a 1ms MPRT display.

[tjbautista01]

The Acer KA220HQ is a TN monitor and has a rather lacklustre response time of 5ms. Your best bet is to use a 1ms MPRT display.

As said above, I tried it again this time connected to a display that can do 59Hz (ViewSonic VA2359-smh) which I also use for playing IIDX on PS2 (via Component->HDMI, plays well almost like playing on CRT) and yeah even on that display the latency is present.

[limyz]

If you are using a playstation controller usb adapter, that significantly adds input lag and should be generally avoided

[tjbautista01]

If you are using a playstation controller usb adapter, that significantly adds input lag and should be generally avoided

Fpr PCSX2, am using keyboard because fully aware of the latency USB adapters have

[refractionpcsx2]

Just to rule it out, can you try this build https://github.com/PCSX2/pcsx2/files/5783396/pcsx2inputlag3.zip This changes Direct3D to use the flip model where as you kinda noted earlier windows uses 3 potential frames of latency by default, this should get around that (but may introduce tearing)

You will need to set your renderer in GSDX to D3D11 in order for it to have any effect. The MTGS Queue option will be missing but it should be forced to 0 in this build. (of course make sure you use one of the included plugins)

That's the last thing I think we can do for input lag.

[tjbautista01]

Just to rule it out, can you try this build https://github.com/PCSX2/pcsx2/files/5783396/pcsx2inputlag3.zip This changes Direct3D to use the flip model where as you kinda noted earlier windows uses 3 potential frames of latency by default, this should get around that (but may introduce tearing)

You will need to set your renderer in GSDX to D3D11 in order for it to have any effect. The MTGS Queue option will be missing but it should be forced to 0 in this build. (of course make sure you use one of the included plugins)

That's the last thing I think we can do for input lag.

Tested the build linked on the ViewSonic screen as well as the previous build you sent me (the inputlag2 one) and although tearing is evident on keyboard the timing is close to spot-on Will test with the Japanese IIDX controller this time with the Super Joy Box Pro adapter I have here (said combination was touted by many players back in the day to be the virtually lagless option when using the controller for PC)

[refractionpcsx2]

okay well you can use the most recent dev build, the main thing is using that GSDX plugin.

Out of curiosity, if you use that GSDX with VSync enabled, what is the lag like?

But if your results are accurate, then windows is the cause of the remaining input lag, not the emulator, the changes in that gsdx is the best we'll be able to do (once it gets merged)

[limyz]

I forgot to mention I was using software rendering in my gameplay. With GSDX with Vsync enabled, I keep hitting into the next timing window (>33ms in GuitarFreaks or >22ms in IIDX). However, these games have adjustable timing in its options if the perceived delay is incorrect.

[refractionpcsx2]

Thanks limyz, but I'm only interested in the D3D results in the test plugin with vsync right now.

[tjbautista01]

All tests done on inputlag3 build and game used is IIDX 7th Style as later styles (HS and above) can compensate for display lag:

Using controller with USB adapter - has lag but I think it's just the adapter (I did say virtually lagless not completely lagless) Enabled vsync - very laggy Using controller with USB adapter and vsync enabled- laggier than playing on actual PS2 connected to an HDTV via composite

For the sake of it I tried software renderer and timing's pretty much the same I think

[refractionpcsx2]

Cool, thanks for testing.

Anyway, as we came to the conclusion before, if it's "PS2-like" I'm happy, if we do better with that plugin, great, because that will get merged at some point. But I'm not going to rush to try and improve things over the original hardware.

[tjbautista01]

So just an update: tried it again this time using a USB IIDX controller and yeah timing is pretty much ok. Usin a controller thru a PS2-USB really did introduce a good amount of input lag. Can't score as well as on PS2 but it's pretty close.

[limyz]

Can you try a much dense chart (any black another will work)

[tjbautista01]

Can you try a much dense chart (any black another will work)

My current skill level is only up to 8s-9s so for things like Another/Black Another you might need someone else to test

[refractionpcsx2]

Is this still an issue when keeping the latency down (Cubeb API) and doing things in order to reduce input lag (like VRR and reducing the MTGS queue size)

[tjbautista01]

the original problem in this thread (backing audio and keysounds desyncing) is no longer an issue as established a few posts ago. the remaining issue is latency from multiple parts (input, display, audio) which i'll touch in here.

input doesnt seem to be an issue as long as vsync is disabled and mtgs queue is 0 so that's crossed out. i have yet to try this on stenzek's qt build with the 'sync to host refresh option' though.

on audio re: cubeb the reported audio latency is a bit too high even on wasapi. compared it with playing iidx on mame and in the latter you hear the sound the moment you press a key (mame's 3.33ms on portaudio wasapi exclusive compared to 10ms on cubeb wasapi even when set to 'use smallest value possible'). not significant for other games but def an issue for rhythm ones.

display latency (settings: d3d11 with weave deinterlacing) as i said earlier is kinda similar to plugging an actual ps2 in a laggy hdtv in that i have to set the in-game timing offset to -2 to achieve better results, which would be difficult for older styles that don't have the offset adjust option but unless there's a way to improve display latency i think this is as 'ps2-like' as it's gonna get?

[Prism019]

the original problem in this thread (backing audio and keysounds desyncing) is no longer an issue

The original problem in this thread is the latency. The audio and keysounds desyncing issue was only ever for IIDX 3rd through 8th styles, as this post stated.

from this issue's original post (emphasis mine):

Description of the issue: Once in a song, there is a set backing track, on top of which the key samples play to form the whole song. Currently in the emulator, this backing track is roughly one beat off from where it should be, as if it started playing ever so slightly late. Unlike the issue with the non interlaced IIDX games, the speed at which the tracks play or the delay between the two doesn't seem to change.