USDX does not handle audio files with more than two channels (stereo) correctly (on macOS and Linux).

bohning commented 3 weeks ago

Actual behaviour

USDX seems to incorrectly handle multi-channel audio files (multi as in more than two channels = stereo), at least on macOS and Linux (but seems to work on Windows).

I have an m4a which contains 6 channels of audio (known as "5.1", "3F2R/LFE" is the info that VLC provides me - meaning "3 front, 2 rear/Low Frequency Effects = bass/subwoofer). When using as audio source via #MP3, USDX plays back the file way faster than it should, so it's not usable. It seems that it tries to squeeze in audio from multiple channels into a timeframe. The playback sound is okay, but the timings are messed up, e.g. the timer is a lot faster than regular time. If there is a video, the video will play with the increased speed.

Expected behaviour

USDX should be able to handle multi-channel audio files, probably by just disregarding the additional channels and using left and right channel for stereo playback.

Steps to reproduce

This 5.1 m4a file can be used together with the following minimal txt to reproduce and debug this issue. The lyrics helper is on steroids, the time display progresses way too fast.

#TITLE:multi-channel bug
#ARTIST:multi-channel bug
#MP3:a-trip-far-far-away_5.1_.m4a
#BPM:400
#GAP:18060
: 0 1 0 
- 1
: 1 1 0 
E

Details

USDX version: latest main (self-compiled)
Operating System + version: macOS Sonoma (14.5), Intel (confirmed to also happen on Linux, but doesn't happen on Windows)

bohning commented 3 weeks ago

I guess the comment here says it all: https://github.com/UltraStar-Deluxe/USDX/blob/80adde2e657d894793dc779847ed049950f29727/src/media/UAudioDecoder_FFmpeg.pas#L444

bohning commented 3 weeks ago

@s09bQ5 Would that be something you could take a look at? Or decide that it's not worth fixing and require a (max) 2-channel audio file?

s09bQ5 commented 3 weeks ago

I'd love to look into this, but I'm quite busy until Sunday. Can you provide a test file? How many channels does your playback device have?

bohning commented 3 weeks ago

There is no rush here whatsoever. I linked a test file above (it's a random 5.1 m4a file). My playback device has 3 channels (center left, center right and subwoofer).

s09bQ5 commented 1 week ago

It's a bug in TAudioConverter_SWResample.Convert. We are telling swr_convert that the output buffer is too small. So we are constantly feeding more samples into the converter that we drain from it and swr_convert happily buffers all the excess samples.

bohning commented 1 week ago

Ah, thanks for looking into this and finding the cause of the issue. Is this fixable, i.e. are you able to fix that?

UltraStar-Deluxe / USDX