Issues with BT and FFMpeg extension

Tolriq commented 4 months ago

Version

Media3 1.3.1

More version details

No response

Devices that reproduce the issue

Android: 10 - 29 [armeabi-v7a,armeabi] / Infinix - Infinix X657C [X657C-GL - Infinix-X657C]

Devices that do not reproduce the issue

Most of them

Reproducible in the demo app?

Not tested

Reproduction steps

Play music when connected to BT on that device when using ffmpegextension.

Expected result

Should work

Actual result

Fails:

2024-04-30 23:08:46.366 Error/ExoPlayer: DecoderAudioRenderer - Audio sink error
q4.t: AudioTrack init failed 0 Config(48000, 12, 1152000) Format(null, null, null, audio/raw, null, -1, null, [-1, -1, -1.0, null], [2, 48000])
    at q4.j0.a(Unknown Source:61)
    at q4.p0.u(Unknown Source:22)
    at q4.p0.y(Unknown Source:164)
    at zy.a.y(Unknown Source:4)
    at androidx.media3.decoder.ffmpeg.b.F(Unknown Source:256)
    at androidx.media3.decoder.ffmpeg.b.y(Unknown Source:99)
    at o4.e0.e(Unknown Source:1017)
    at o4.e0.handleMessage(Unknown Source:308)
    at android.os.Handler.dispatchMessage(Handler.java:103)
    at android.os.Looper.loop(Looper.java:264)
    at android.os.HandlerThread.run(HandlerThread.java:67)
    Suppressed: q4.t: AudioTrack init failed 0 Config(48000, 12, 1000000) Format(null, null, null, audio/raw, null, -1, null, [-1, -1, -1.0, null], [2, 48000])
        at q4.j0.a(Unknown Source:61)
        at q4.p0.u(Unknown Source:96)
        ... 9 more
    Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
        at android.media.AudioTrack$Builder.build(AudioTrack.java:1018)
        at q4.j0.b(Unknown Source:60)
        at q4.j0.a(Unknown Source:4)
        ... 10 more
Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
    at android.media.AudioTrack$Builder.build(AudioTrack.java:1018)
    at q4.j0.b(Unknown Source:60)
    at q4.j0.a(Unknown Source:4)
    ... 10 more

2024-04-30 23:08:46.367 Error/ExoPlayer: DecoderAudioRenderer - Audio sink error
q4.t: AudioTrack init failed 0 Config(48000, 12, 1000000) Format(null, null, null, audio/raw, null, -1, null, [-1, -1, -1.0, null], [2, 48000])
    at q4.j0.a(Unknown Source:61)
    at q4.p0.u(Unknown Source:96)
    at q4.p0.y(Unknown Source:164)
    at zy.a.y(Unknown Source:4)
    at androidx.media3.decoder.ffmpeg.b.F(Unknown Source:256)
    at androidx.media3.decoder.ffmpeg.b.y(Unknown Source:99)
    at o4.e0.e(Unknown Source:1017)
    at o4.e0.handleMessage(Unknown Source:308)
    at android.os.Handler.dispatchMessage(Handler.java:103)
    at android.os.Looper.loop(Looper.java:264)
    at android.os.HandlerThread.run(HandlerThread.java:67)
Caused by: java.lang.UnsupportedOperationException: Cannot create AudioTrack
    at android.media.AudioTrack$Builder.build(AudioTrack.java:1018)
    at q4.j0.b(Unknown Source:60)
    at q4.j0.a(Unknown Source:4)
    ... 10 more

The dump contains:

05-01 12:13:12.471  1000   480  1268 I BufferQueueProducer: [app.symfonik.music.player/app.symfonik.ui.MainActivity#0](this:0xb1663000,id:16642,api:1,p:6695,c:480) queueBuffer: fps=51.44 dur=1010.79 max=92.12 min=11.33
05-01 12:13:12.477 11580  6695  6774 E IAudioFlinger: createTrack returned error -12
05-01 12:13:12.477 11580  6695  6774 E AudioTrack: createTrack_l(-1946121216): AudioFlinger could not create track, status: -12 output 0
05-01 12:13:12.477 11580  6695  6774 E AudioTrack: set(): createTrack_l fail! status = -12
05-01 12:13:12.477 11580  6695  6774 E AudioTrack-JNI: Error -12 initializing AudioTrack
05-01 12:13:12.477  1000   468 11829 W IMGMemtrackHAL: hal_get_memory: memtrack cache rebuild was required
05-01 12:13:12.477 11580  6695  6774 E android.media.AudioTrack: Error code -20 when initializing AudioTrack.

I can ask the user if he allows me to send you the full dump if it's needed.

As a side note I've also reported https://github.com/androidx/media/issues/787 that was completely ignored, but is in fact maybe related, the devices who have that other issues also only have it when using the ffmpeg extension.

Media

Seems to happen on most media for that device.

Bug Report

[ ] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

tonihei commented 4 months ago

Config(48000, 12, 1152000)

This looks like it might be failing because the audio track buffer size is too big (> 1MB). The suppression also indicates the DefaultAudioSink automatically retried with a 1MB buffer but still failed.

As far as I see, the error code "-12" from AudioTrack also means NO_MEMORY, pointing in the same direction.

Are there any other tracks playing at the same time or any other sort of memory pressure? Alternatively, you could retry with an even lower audio track buffer somehow, but it feels like if a 1MB buffer is failing then slightly smaller buffers might fail too.

Tolriq commented 4 months ago

@tonihei not using ffmpeg and not making any other change does fix it for the user.

(I have an option that just change the order of the factory)

This is the same for the other linked issue, not using ffmpeg fixes it.

Seems there's something different the ffmpeg does that can impact BT on some devices but I'm clueless.

If you have ideas of hidden stuff on the extension I can try that last users is more cooperative so I might get him to run tests.

tonihei commented 4 months ago

The only reason I can imagine at the moment is that using FFmpeg causes too much memory to be used, leaving not enough memory for the AudioTrack. I'm not sure if this is a reasonable hypothesis, but it could be related to the fact that both need to allocate native memory outside of the Java layer.

Tolriq commented 4 months ago

That would not explain the other issue that looks similar, I'm currently trying to read code and add some logs on my custom build to help move on those as it's a pain for my users.

I've triple checked and both paths are using the exact same sink:

val audioSink = DefaultAudioSink.Builder(context)
            .setAudioProcessorChain(
                DefaultAudioSink.DefaultAudioProcessorChain(
                    SilenceSkippingAudioProcessor(),
                    SonicAudioProcessor(),
                    replayGainProcessor,
                ),
            )
            .setEnableAudioTrackPlaybackParams(true)
            .setEnableFloatOutput(hires)
            .build()
        if (haveRApi) {
            audioSink.setOffloadMode(offloadMode)
        }

It's just a matter of the order of creation to pass from one case to the other

override fun createRenderers(
        eventHandler: Handler,
        videoRendererEventListener: VideoRendererEventListener,
        audioRendererEventListener: AudioRendererEventListener,
        textRendererOutput: TextOutput,
        metadataRendererOutput: MetadataOutput,
    ): Array<Renderer> {
        return if (preferInternalDecoder) {
            arrayOf(
                getFfmpegRenderer(eventHandler, audioRendererEventListener),
                getDefaultRenderer(eventHandler, audioRendererEventListener),
                MetadataRenderer(metadataRendererOutput, eventHandler.looper),
            )
        } else {
            arrayOf(
                getDefaultRenderer(eventHandler, audioRendererEventListener),
                getFfmpegRenderer(eventHandler, audioRendererEventListener),
                MetadataRenderer(metadataRendererOutput, eventHandler.looper),
            )
        }
    }

Reading internals I think the real cause may be hidden and those devices returns a wrong error code and it's not an issue with memory but just some parameters passed to the sink that will then generate different AudioTrack settings.

Specially since both DecoderAudioRenderer and MediaCodedcAudioRenderer do not create exactly the same the outputFormat passed down.

To be sure I'll add some debug logs at start of DefaultAudioSink.configure() to log the exact full params and see if there's differences with and without ffmpeg for that user.

Do you think about something else that could be useful?

And @tonihei since this is really low level, can you clarify if the AudioTrack itself is somehow related to BT and how? Or it's it's before and then the fact that playing outside BT is working gives another clue. Like a missing event somewhere that prevent releasing the AudioTrack.

tonihei commented 4 months ago

To be sure I'll add some debug logs at start of DefaultAudioSink.configure() to log the exact full params and see if there's differences with and without ffmpeg for that user.

That sounds useful to check.

can you clarify if the AudioTrack itself is somehow related to BT and how

AudioTrack is related to BT in that it handles the entire output path of the audio data. So it will behave differently depending the path and also the capabilities of the output device will be different.

androidx / media