Buffer Issues Detected with Multichannel RSTM Streams

[gheskett]

I've found that the final block in a multichannel stream will often fail to load properly during playback. My best example to demonstrate the issue is with Toad's Factory from Mario Kart Wii (STRM_N_FACTORY_N.brstm). The first stream loads 1280 samples of...something in the last block followed then by what should have loaded, but offset 1280 samples. Because of this, it also compromises the stream's loop. Here's an image comparing BrawlBox's output to what it should look like:

Additionally, not sure if this is technically a bug, but switching between substreams in a multichannel stream is not seamless at all. Judging by the source code, it may or may not be related to pausing the audio and updating the buffer before resuming. A solution to this might just be adding to the buffer that has already been generated; rather than completely rewriting the buffer, let what's already been loaded still play back while appending the newly selected stream to the end of it. This way, there is no need to pause the audio, even if that means the streams don't transition instantly (unless it takes a notable amount of time for them to make that transition; then I wouldn't make that change).

EDIT: This first issue is bizarre. I've found with PCM16 streams, part of the stream one substream up is loaded in place of the stream that should be loaded for a short period (with exception to the last substream in the file for some reason, but Stream 1 still loads part of the second to last substream, so in this case, Stream 1 of Toad's Factory is reloading a different part of...itself?). ADPCM doesn't seem to do this, but its loops still contain static. Perhaps I should provide an example: Animal Crossing City Folk - Museum

[libertyernie]

I think I've fixed the problem with PCM16, but it only affects PCM16, not ADPCM.

Does the ADPCM issue exist when exporting to .wav, or only in playback?

[gheskett]

Huh, so the PCM16 issue appears to be resolved, and it turns out the issue I had with the ADPCM is likely both a fault with VGAudio (which is what I use when I need streams with more than 2 channels) and also a fault by Nintendo themselves for a bad loop on their part. So the first issue is most likely resolved completely; sorry about the false alarm with ADPCM.

[Thealexbarney]

@gheskett What kinds of issues were you seeing with VGAudio's conversions?

[gheskett]

@Thealexbarney I don't know exactly what's going on, but I think it has something to do with the shifting of loop points. With BrawlBox, if you input a starting loop point that isn't a multiple of 14336, it will then shift the loop to a multiple of 14336, and the audio in between the input starting loop and new starting loop points will be tacked on after the initial end loop point of the stream. If my observations are correct, VGAudio shifts the loop points in this case, but it does not tack on the audio after the starting loop point to the end like BrawlBox does. This can easily lead to a bad loop and sometimes cause an exception to be thrown if there isn't enough extra audio at the end of the input file to compensate for that shift. While I'm not sure if that qualifies as a bug, it feels like an oversight.

[Thealexbarney]

@gheskett Thanks for the info. I just did some testing on 1 and 4-channel ADPCM BRSTMs back through version 2.0.0-beta1, and VGAudio correctly copies the data when aligning the loop. Did you actually hit an exception when there wasn't enough audio past the end loop? If you happen to run into the issue again, it would be great if you could let me know so I have a sample to go off of.

[gheskett]

@Thealexbarney I'm a little embarrassed to say the points that I made above were all incorrect. It has been awhile since I posted that last bug, so I don't remember exactly what was going on, but I took a guess at it since I wasn't in front of my computer at the time of posting. My conclusions felt like they made sense at the time because I had run into an exception like that before, but now that I think about it, I may have given an input too large when setting loop points one time and just remembered that incident incorrectly (though I would recommend maybe setting the default end loop point to the number of samples in the stream for the GUI interface when a user wants to loop a WAV or maybe setting the end loop point to the number of samples in the stream if the input is too large).

All that being said, the static issues that I ran into were definitely present back when I made that post in November. I was just messing around with some audio trying to replicate what was going on, and I encountered this:

The waveform at the top represents the original waveform and the middle one represents the result after making a BCSTM conversion with VGAudio. The bottom waveform represents the difference between the two, and the split in the audio represents the end loop point of the BCSTM (not the original). As you can see from the bottom waveform, there is minimal loss present throughout the conversion (as a natural result of ADPCM compression) up until the last 10 samples before the end loop point. While these samples may not seem like a big deal and require zooming in to even see, they are audible in the result. This issue isn't greatly represented here, but the one I dealt with back in November was a lot worse.

[Thealexbarney]

Moved the VGAudio-related issue to Thealexbarney/VGAudio#91 to keep this one cleaner