DSTWO sound output causes FPS to go insane

[Nebuleon]

Per above.

I've tested this in a few games, including my fave, Super Mario World. Complex frames, with lots of OBJs and BGs, are drawn out to the screen, but then sound output via the DSTWO slows down the game to about 4 FPS.

Disabling the sound output, by emptying S9xGenerateSound and S9xProcessSound (before commit c01a2a42168695233ecc69c4a60ed918e7701fb9 (2012-12-28)) or by going into the Video & Audio settings to disable sound (after that commit), gets the frame rate to acceptable levels without deviation.

Editing source/nds/ds2sound.h to use 11025 Hz audio oddly improves the FPS to become very acceptable, but deteriorates the sound severely (obviously). 22050 Hz and up have this problem.

a) It may have to do with the reduced buffer size. However, setting 22050 Hz and up to use a reduced buffer size causes underflows. b) I've tried to use code similar to the DS2SDK Example Plugin's src/wave.c to replace S9xProcessSound. That code can safely assume it can read more waveform data ahead of time, for 16 buffers worth of 2048 samples (32768 samples ~ 0.75 second). When used in Snes9x, it causes underflows and glitches.

[Nebuleon]

Fixed in CATSFC 1.16. Open new issues if you have this problem with a specific game, one issue per game.

[Nebuleon]

Unfixed in CATSFC 1.18. CATSFC 1.16-1.17 crashed after 5-15 minutes of emulating a game because of the audio being pushed in a timer interrupt. I reversed this to prevent the crash, so this issue appears again.

[HitsuMaruku]

@Nebuleon, I write this in the hope that it helps you develop a solution to this problem.

In my search for an updated version of NDSSFC, I came across both BAGSFC and this. I found, however, a similar issue with sound skips. It seems that some calls to sounds are simply lost (e.g. a melody where every two or three notes is simply missing). I presume that somehow certain audio calls are simply being lost in transit, exactly the same way that BAGSFC lost certain controller calls, and the call would simply not go through.

I've tested extensively using different versions (as found and retrieved from filetrip.net), and the issue comes into play when moving from ShadauxCat's v1.1 to your v1.08. Based on that jump, I theorize that the issue being encountered may lie somewhere within the changes you first made when taking over the project. It may be possible to simply find the underlying cause and undo that particular change, and mirror it to the current version (though I do understand that drastic revisions have been made, and that it may, in fact, not be possible to do so).

My current setup is as follows: Hardware: DS Lite, Supercard DSTwo, MicroSDHC 4GB Class4 C04G TAIWAN (unsure of manufacturer) ROM tested: Final Fantasy IV (Japanese native, revision 1.1, patched with fan translation) Audio issues: Skipping partial audio calls (issue does not exist in NDSSFC, BAGSFC or CATSFC v1.01 and v1.1) Graphical issues: Slight decrease in frame rate (from NDSSFC v1.05), but may be attributable to other causes Controlling issues: None

I hope that this information helps you to find the cause of the issue that is being experienced by myself and others in the community.

EDIT: Followup: I reverted the initial changes made to "source/nds/entry.cpp" (changed Settings.SoundSync to FALSE) and recompiled. This, for me at least, fixed the issue. There is a very slight decrease in performance at times, but it is barely noticeable when it does happen (may be more noticeable for an action-type game as opposed to an RPG). For the moment, I'm content with this short-fix. For the games that I play, I'd rather full sound and intermittent 90% speed than full speed and constant sound loss. Please let me know if there might be anything else that I could do to help fully resolve the disparity between speed and sound.

PS: Thank you very much for all of the work you've put into the project and for keeping it open-sourced. The new Display Mode 4 makes games especially more enjoyable.

[Nebuleon]

@HitsuMaruku: Thank you for your analysis of the problem. Owing to various changes made to audio after 1.08, more particularly at and after 1.16, I'm not sure whether removing Settings.SoundSync = TRUE; would make the sound any better. For me, the earliest version of CATSFC, CATSFC 1.1 (1.07), skipped notes and jumps in Super Mario World, Yoshi's Cookie and Legend of Zelda. Adding Settings.SoundSync = TRUE; fixed that.

Without SoundSync, sound samples are mixed all at once with the current notes at the end of a frame. If a note was shorter than a frame, or the note's "attack period" has passed, then it's lost. With SoundSync, after each scanline (224 in a frame), some samples are mixed for 1/13440th of a second.

At 1.16 and before 1.18, I use the timer interrupt from the DS2 SDK in ds2_timer.h to push sound, and it's much fuller (with SoundSync) than before, improving smoothness of picture rendering as well. Without SoundSync, there are too many samples to mix at once, it has no samples to emit at the end of the previous buffer, and you hear silence which turns into crackling.

I reverted the timer interrupt audio changes in 1.18 because of a crash in the menu which didn't occur before 1.16. But after 1.20, in commit 582b3a23ceeb9e894b0f1d43de09e97a91a4d3b9 (2013-01-15) I discovered this gem:

3417 | choose_menu(&main_menu); which goes to: 3370 | if(NULL != current_menu) { 3371 | if(current_menu->end_function) 3372 | current_menu->end_function(); 3373 | } which references a new instance of this variable: 1668 | MENU_TYPE *current_menu;

Essentially, each time you enter the menu, the main menu is chosen, and whether or not choose_menu executes the "previous menu" (which should have been NULL all along)'s end_function() depends on the uninitialised value of current_menu. So the emulator worked by chance all this time, and my audio interrupt code got bitten by this randomness.

As for validating audio optimisations, I've made a proof-of-concept .spc file player which isn't ready for public use yet. It struggles to emit full sound at 22050 Hz even with ds2_setCPUclocklevel(13), crackles, has problems with echo, and crashes often. However, it's how I found the uninitialised-current_menu-before-choosing-the-main_menu bug. Without the initialisation, this .spc player crashed every single time.

[HitsuMaruku]

Well, I'm forced to admit that I don't know nearly as much as I'd like to regarding the platform or SDK, but what you're saying makes sense to me. It sounds to me that the menu calls became a memory leak, and that's why it would crash after several minutes of gameplay (just speculation). I'll see about recompiling 1.16/1.17 with the changes you made to initialization and see if it yet crashes on me (unless you happen to have your own release by then, of course). Looks like a busy weekend ahead of me, so I might not actually get to it for a while.

Regarding the .spc player, I regrettably don't think I know enough regarding the way such emulation works, but could it be that the Settings.ThreadSound (just above Settings.SoundSync) must be True in order for sound to be threaded onto the CPU? Again, only speculation from a novice on the subject, but I state it in hopes that it may point in the right direction (and also that I may learn something from it).

PS: Removing Settings.SoundSync=TRUE (or, more specifically, changing it to FALSE) happened to resolve the issue for FF4J, but it's the only game that I've tested thus far. It may simply be an issue with that particular game, since you seem to have the reverse effect on other games. Perhaps there could be some method of changing the sync option during runtime depending on the game being played (or even manually as an option in menu), unless it's something that's initialized during startup (in which case, you can ignore this spiel, haha).

[Nebuleon]

SoundSync is indeed initialised during startup.

As for ThreadSound, that's for multi-core computers; it tells a second "core" to run a separate thread of execution to mix the sound.

I'm retesting the code from 1.16 with initialisations from commit 582b3a23ceeb9e894b0f1d43de09e97a91a4d3b9 right now, actually. It seems to disable the upper screen sometimes, but sound continues to run. Touching the lower screen in this state induces a crash.

edit: this crash occurred only once and I can't reproduce it... maybe it was a fluke?

re-edit: the crash occurred again in Super Bomberman 5 with timer-interrupt audio and the fixes introduced in commit 51e18ffb2fa40a3dbb28ddb9789bbf0e2665132a and others. This time, the upper screen froze, and sound stopped.

[Nebuleon]

SoundSync is essential with timer-interrupt audio (1.16-1.17); without it, no audio is ever mixed.

[Nebuleon]

I looked at this again. Commits 58948ffd1d05084f3d84bc12a2e75144741b292c, 30fe9eaf6ab8b0701419949a4415736675d15377, e61731a524028f8286f83a82686b2f4e236c1a9d and d4dd98e8c180532f24de342482e54f28874f06ef (2013-02-01) (v 1.25) should improve matters a lot.

In 58948ffd1d05084f3d84bc12a2e75144741b292c, I make S9xProcessSound, which is called once per line (224*60 times per second), send its audio only every 23 milliseconds. Previously, it attempted to send 92 milliseconds of audio every frame.

This added a lot of crackling and odd silence, so in 30fe9eaf6ab8b0701419949a4415736675d15377 I arranged for delays in frameskip code to produce sound instead of idling.

Commit e61731a524028f8286f83a82686b2f4e236c1a9d reverts 2b715684087675747df7cb8995695936f20c782b, which I thought solved the problem - but it didn't.

And finally, commit d4dd98e8c180532f24de342482e54f28874f06ef adds an option in Video & audio, called Fluidity, that you can use to set the value of Settings.SoundSync per-game. With SoundSync and the new sound timing, you can get pretty close to SNES sound hardware, but it consumes massive CPU time and can delay images by up to 166 milliseconds. Without SoundSync, you can delay notes in audio up to 23 milliseconds. The default is SoundSync = FALSE, which makes many games playable. "Soundtrack games" can use the new "Fluidity / Prefer fluid audio" setting.

Please test and provide feedback on the fix.

[HitsuMaruku]

I've been keeping an eye on this since I last posted, and while I've found that I don't know nearly enough to help with coding, I can indeed help with testing.

I've been recompiling from source with SoundSync=FALSE for comparison with each release, and as far as I can tell, the new option that you've implemented does that for me during run time. I still have the same issue, unfortunately, with skipping audio calls with the setting Prefer Audio (which I believe is SoundSync=TRUE, yes?) and seems to run fine with Prefer Video (which, oddly enough, provides better audio for me than with Prefer Audio). Again, this could simply be the game I'm playing (FFIV), but it really does help to have the option. Very nice idea!

I also still encounter the issue of missed input calls (e.g. pressing up 4-5 times too quickly and the emulator only recognizing 2-3 calls) on either Preference setting, but I think that might be listed on another issue page.

[Nebuleon]

Controller status synchronisation was perfect in versions 1.16 and 1.17, but it was a fluke because those versions crashed. They sent audio during a timer interrupt, and the DSTwo doesn't like that.

Unfortunately it's something we must live with, because there's no way to get interrupts when the controller status changes. And if you happen to press your buttons while the image is rendering, or the audio is rendering, these buttons are indeed skipped. The controller status is only examined while the SNES CPU is being emulated.

Maybe the DSTwo would be stable enough to look at controller status during a timer interrupt, though... One that would be fired, say, 120 times per second. But that's another issue! If there's no issue page for it, please create one.

As for the SoundSync = FALSE providing better sound than SoundSync = TRUE... you now know how I felt trying to tweak the setting in the first place (in v 1.08). It's really weird, but now you get the option to change it per game so you can control the speed yourself.

Finally, I'd like to say you might not code, but knowing how to compile stuff makes you not an ordinary user/tester :)

[Nebuleon]

Nope; looking at the controller status in a timer interrupt also freezes the emulator. Much faster than timer-interrupt audio, in fact.

[Nebuleon]

Refixed in CATSFC 1.26.

[HitsuMaruku]

Under testing for v1.28, I have similar results as before with Prefer Fluid Video: no sound loss; however, with Prefer Fluid Audio, I have much less sound loss than I did in previous versions. I think I skipped over 1.27, so I'm not certain which version helped. Regardless, setting Prefer Fluid Video fixes the problem completely (for my game) with no video, sound or input loss, so attempting to fix it would be a moot point. Very nice work!

And I appreciate the vote of confidence. =) I don't code much yet, but I start my first post-grad job in two weeks, so hopefully I will be much better someday.