SPU2ghz (patch) : Completely new Timestretching implementation (work-in-progress)

[GoogleCodeExporter]

This is my first draft patch for a compeltely new and redone timestretcher
for SPU2ghz.  First the broken stuff:

 * XAudio2, DSound 5.1, and ASIO modules are broken.  But then again none
of them worked too well anyway.  They will get fixed (well, except for
ASIO, which I don't care to mess with), but for now they're not important.

 * Surely some other stuff I can't think of atm.

The changes and benefits are numerous, so I'll just give a basic summary:

 * Greatly reduced average latency.  For games that run well, latency can
now be below 60ms (nearing the point of being unnoticeable!). 

 * Many minor optimizations to the SndOut pipeline - nothing significant
however.

 * SndOut pipeline now retains audio accuracy slightly better when using
timestretching.

 * Improved SoundTouch configuration settings.  Overall audio quality
should be better now, and a little faster too when Timestretch is enabled.

 * Revamped Configuration screen.  New options, new descriptions, etc.

 * Added a Volume Boost option, which can be useful for people using
soundcards with low-gain line outs (like my SB Live! 32 External). 
WARNING!  Volume Boost can make some games sound very bad if they are
already louder than usual.

 * Implemented configuration options for Waveout module.

 * Fixed a bug in the ADPCM cache that could lead to memory corruption.

 * Fixed a potential race condition in the SPU2 framelimiter.

That should about cover the important stuff.  I'll work on getting dsound51
and xaudio2 converted to the new output module system and upload a new
version of the patch when those are done.  In the meantime the
timestretching code needs tested on a wider variety of games and such, and
the new low-latency dsound driver needs some reps on Vista machines. :)

Note: You can enable overrun warnings and timestretching statistics by
turning on the "Buffer Overruns" debug message in the Config panel.

Original issue reported on code.google.com by Jake.Stine on 26 Oct 2008 at 9:18

Attachments:

• spu2ghz-r239-timestretcher.patch

[GoogleCodeExporter]

I'm on it, thanks again :)
Btw, gigaherz will definatly hate the broken ASIO :D

Original comment by ramapcsx2 on 27 Oct 2008 at 10:03

[GoogleCodeExporter]

Yeah, I imagine.  It looks like he worked *really* hard on it.
But it's doomed to never really work right anyway, I'm afraid.  ASIO was never
intended for games.  When ASIO calls itself "low latency" it's in the 
professional
midi/recorder sense... because in midi environments you need things to be timed 
to
the *microsecond* because you're usually outputting sound into a secondary 
external
mixer.  If sound isn't timed exactly right, the mixing process will flunk.

In the world of games, of course, we're working in milliseconds, and in PCSX2
emulation specifically we're working in sets of 20 ms (~50/60fps).  The "kernel
latency" that ASIO bypasses is only like a couple milliseconds at worst, which 
means
that even if the driver was functioning perfectly it would reduce latency by a
whopping 1% over what's needed just to keep the timestretcher happy.

By the way, I've almost got a new patch ready.  I tweaked the timestretch 
manager a
bit and fixed up DSound 5.1, and changed one of the default conf values.  I also
streamlined SndOut a wee bit more (yet again!).  No performance boost this 
time, but
it's more "ready" for proper-like SSE optimizations, for when those happen to 
get
implemented.

Oh yeah, I should mention that due to the number of config changes I opted to 
use a
separate ini file.  It was too much work to try to be backward compatible.  So
spu2ghz will use spu2ghz-v2.ini instead of spu2ghz.ini.  Saves people the 
trouble of
having to delete their old ini to avoid crashes.  (although the code also 
checks the
sanity of most values now so bad configurations shouldn't cause it to die 
anymore)

Original comment by Jake.Stine on 27 Oct 2008 at 10:19

[GoogleCodeExporter]

Ok, first testing showed yet again how Vista can kill all efforts ><
On my machine with Vista 32 I have to use at least 5 dsound buffers and a 
latency of
120ms.
Starting from that latency it all works like a charm though, great job :)

Well, maybe its possible to down the stretcher usage a little more ^^

* SPU2 > Timestretch Stats > 17% of packets stretched.
* SPU2 > Timestretch Stats > 16% of packets stretched.
* SPU2 > Timestretch Stats > 56% of packets stretched.
* SPU2 > Timestretch Stats > 15% of packets stretched.
* SPU2 > Timestretch Stats > 18% of packets stretched.
* SPU2 > Timestretch Stats > 23% of packets stretched.

This is a log of Xenosaga2 running a menu screen at 60fps.
I'll try and see if I can make it work a little less ;)

Original comment by ramapcsx2 on 27 Oct 2008 at 10:31

• Changed state: Accepted

[GoogleCodeExporter]

The 120ms latency isn't unusual.  That's one of the things I had to change. 
Defaulting to 60 was way too low for most games.  Actually it should be able to 
run
with 90 or 100 once I fix this glitch in the stretch manager.  The math is 
skewed --
it can't raise tempo as fast as it needs to, and so buffer overruns happen too 
often.
 And at least the dsound buffer systems works at 5.  Even with 5 buffers it's still
1/2th the latency of the old implementation.

I could put in a Vista OS version check and automatically default to 5 buffers 
for
Vista users.

As for the timestretch stats... good luck. The more you expand the values to 
reduce
use of the stretcher, the less able it is to cope with FPS fluctuation. :(  
Maybe
things will be better when I fix the stupid part of this math that makes it 
overrun
too much.

Original comment by Jake.Stine on 27 Oct 2008 at 10:48

[GoogleCodeExporter]

Well, we could add a system much like the old stretcher, where it would look at 
past
experience and reduce the stretcher usage. This would prolly bloat the code 
again though.
I'll just have to see how this evolves further first, then maybe add it :p

So far its looking good though, keep at it ;)

Original comment by ramapcsx2 on 27 Oct 2008 at 10:55

[GoogleCodeExporter]

I don't think past experience-based stuff works, honestly.  It's one of those 
things
that sounds good on paper but I've never seen one implemented that was actually
capable of delivering.  The problem really is that as soon as you start to make
assumptions based on past experience you're going to leave yourself open to
surprises.  And PCSX2 is still full of FPS surprises. :P

Also!  The ability of the stretcher to both avoid skips *and* stretch cleanly 
(with
less sudden tempo changes) is exponentially proportional to the buffer size.  A
buffer size of 160ms is nearly infallible in my tests, even with the bum math 
that
can't avoid overruns very well.  The good news?  160ms is still less than 
before. :D

Original comment by Jake.Stine on 27 Oct 2008 at 11:05

[GoogleCodeExporter]

Could you give me an aproximation on how big the delay in the old system was at 
1024
* 8? :p

Original comment by ramapcsx2 on 27 Oct 2008 at 11:14

[GoogleCodeExporter]

Old system was actually 4096 bytes per buffer (1024 samples, stereo, 16 bit).
New system is 2048 (512 samples, stereo, 16 bit).  It can get confusing because 
of
the stereo part.  It's not always commented when a sample buffer is stereo or 
not.

Anyways, latency calculations:

LatencyInSeconds = BufferSizeInSamples / 48000;

So each buffer took roughly 0.02 seconds, or 20ms.  Eight buffers would have 
been
160ms.  Under the new system it's all divided by half.  So roughly 50 ms for 5 
buffers.

And for what it's worth I can run 2 buffers on dsoundOut and 3 buffers on 
waveOut,
and it never skips a beat.  XP for teh win!

(but then again xenosaga runs at about -10 fps.  Yes, I'm pretty sure it runs
backwards at times.. -_-)

Original comment by Jake.Stine on 27 Oct 2008 at 11:28

[GoogleCodeExporter]

Yeah, the game has too many texture swizzles with dx9 (I think that was the 
problem).
Try the software renderer ;)

Original comment by ramapcsx2 on 27 Oct 2008 at 11:42

[GoogleCodeExporter]

Improved timestretching logic.
Patch attached, made against r229.

This should really improve stretch quality big time, and it should help you get 
much
better non-stretch ratios too.  I didn't fiddle with the no-stretch zone yet... 
but
I'm guessing this version should be able to handle a bigger range of buffer and 
tempo
zones for nominal (1.0) output without over/underrunning.

DSound 5.1 is still broken though.  I have a buffer size mismatch somewhere in 
the code.

Original comment by Jake.Stine on 27 Oct 2008 at 2:26

Attachments:

• [spu2ghz-r239-timestretcher v2.rar](https://storage.googleapis.com/google-code-attachments/pcsx2-playground/issue-31/comment-10/spu2ghz-r239-timestretcher v2.rar)

[GoogleCodeExporter]

Eh, I may have made the timestretcher too aggressive.  Ah well, it's close 
enough for
now.  I may tone the aggressiveness back a little bit later on though.  If the 
TS
tries too hard to keep audio playback buffers nominal it can cause wrrbly sound 
on
games with inconsistent FPS.  So it's a matter of striking a good balance 
between
keeping buffers nominal (ideal for good smooth games) and allowing a little 
wobble
room for smoother stretching.

Original comment by Jake.Stine on 27 Oct 2008 at 2:54

[GoogleCodeExporter]

"Eh, I may have made the timestretcher too aggressive"
Yes, its also too "afraid" now, constantly stretching +1 or -1 % even with 
superbig
buffers at 60fps.

The system as a whole is working cool though, its very robust to speed changes.
Just a wider no-stretch zone is missing imo :p

Original comment by ramapcsx2 on 27 Oct 2008 at 5:34

[GoogleCodeExporter]

Improved timestretcher!
Fixed DSound 5.1!
Some other things (bugfixes).
Sorry I can't say more in detail.. gotta run.

Patch is against r245.  Hope it works for ya as well as it works for me!

Original comment by Jake.Stine on 28 Oct 2008 at 7:32

Attachments:

• spu2ghz-stretch.r245.patch

[GoogleCodeExporter]

Crap, no time as well ><
I'll test it first thing tomorrow ;)

Original comment by ramapcsx2 on 28 Oct 2008 at 7:45

[GoogleCodeExporter]

*Newer and better timestretcher!*
This one is damn near infallible.  Actually, it's still a little weak against
underruns on games that run at low FPS... but then again by the nature of 
latency
that's kinda to be expected (it's hard to compensate when a game's FPS is 
actually
slower than the latency buffers are long ;).  In trials, setting the latency to 
160ms
pretty well killed all overruns and underruns on games that typically ran 20-40 
fps.
 Latencies of 100ms were still quite playable, but would experience an underrun every
minute or so (which was actually unnoticeable except for the console log 
letting me
know it happened :P)

Also in this pach:

 * Numerous stability fixes to the dsound drivers (mainly concerning emulation resets
and such)
 * Bugfixes in the config load/save that could cause dsound to crash in rare
occasions sometimes.
 * fixed a couple small memory leaks (again on emu resets).

Added Bonus:

 *** Working XAudio 2 Drivers ***

There's no configuration options panel for it yet, and I'm not sure how much of 
one
it will need for that matter.  Right now it's configured for OPTIMAL 
performance ..
the bare minimum possible values for latency.  Let's see how it runs on Vista
machines at that setting and take it from there.

Patch is against r245.

Original comment by Jake.Stine on 29 Oct 2008 at 5:39

Attachments:

• spu2ghz-stretch-v2.r245.patch

[GoogleCodeExporter]

Ok, this is great stuff :D

XAudio2 works here on Vista, and with the new stretcher I can safely set a 
delay of
120ms to almost never get skips.
At 90ms (with the occasional skip at high fps fluctuation, barely disturbing) 
the
delay almost feels like realtime.

The new safe zone for fullspeed games is 100% working :)
I got a log where it never stretched aa single packet in 5 minutes runtime.

Well, I got no idea how to make that even better. Congrats :)

Original comment by ramapcsx2 on 29 Oct 2008 at 1:17

[GoogleCodeExporter]

Well, I'd like to try XAudio2 output as soon as I can get that DLL from 
re4rainbow... :-)

Original comment by eliotfur on 29 Oct 2008 at 1:55

[GoogleCodeExporter]

//Well, I got no idea how to make that even better.//
There is still exist a FFXII issue :o

Original comment by zantezuken@gmail.com on 29 Oct 2008 at 4:42

[GoogleCodeExporter]

Patch applied in r247.  Issue resolved.

Original comment by Jake.Stine on 31 Oct 2008 at 12:54

• Changed state: Fixed