DefaultAudioSink: arguments out of range on specific content

DavidFair commented 2 years ago

On random media setting PlaybackParameters has no effect, this was noticed in: https://github.com/jellyfin/jellyfin-androidtv/issues/1517 I've hit the limit of the debugger as there's something within the OS layer that I can't step into. Looking around other developers have reported this exception was thrown when the buffer passed to AudioTrack was too small, but as the playback speed is half then naively I'd assume the buffer effectively serves 2x the content now.

Since we don't directly interact with AudioTrack in any-way within the app I'm pretty sure this is something ExoPlayer isn't handling correctly:

Reproducer

Ensure the DefaultRendererFactory has setEnableAudioTrackPlaybackParams(true);, this makes the reproducer appear in the debugger (rather than be handled elsewhere)
If the above is set to false the playback speed still ends up forced to 1x, but I can't see where it's failing within the debugger
Call exoplayer.setPlaybackParameters(0.5)

On working media:

The following gets called, it does not throw an exception: https://github.com/google/ExoPlayer/blob/8129675b065bc98330ec775fb7a538c139793070/library/core/src/main/java/com/google/android/exoplayer2/audio/DefaultAudioSink.java#L1522
The updated parameters are reflected in the new audioTrackPlaybackParameters with speed set to 0.5

On "broken" media:

The above LOC gets called

The following stack trace is thrown and caught in the same try-catch block:

W/DefaultAudioSink: Failed to set playback params
  java.lang.IllegalArgumentException: arguments out of range
    at android.media.AudioTrack.native_set_playback_params(Native Method)
    at android.media.AudioTrack.setPlaybackParams(AudioTrack.java:2109)
    at com.google.android.exoplayer2.audio.DefaultAudioSink.setAudioTrackPlaybackParametersV23(DefaultAudioSink.java:1319)
    at com.google.android.exoplayer2.audio.DefaultAudioSink.handleBuffer(DefaultAudioSink.java:753)
    at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.processOutputBuffer(MediaCodecAudioRenderer.java:639)
    at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.bypassRender(MediaCodecRenderer.java:2124)
    at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:773)
    at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:982)
    at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:486)
    at android.os.Handler.dispatchMessage(Handler.java:103)
    at android.os.Looper.loop(Looper.java:214)
    at android.os.HandlerThread.run(HandlerThread.java:67)

audioTrackPlaybackParameters is updated to 1.0 speed internally so the user's selection ends up "ignored"

Media info

Working example:

ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Big_Buck_Bunny_h264_10s_30MB.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : Big Buck Bunny, Sunflower version
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    composer        : Sacha Goedegebure
    encoder         : Lavf57.63.100
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    genre           : Animation
  Duration: 00:00:10.00, start: 0.000000, bitrate: 24563 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 24559 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler

Failing example:

ffprobe version 4.2.4-1ubuntu0.1 Copyright (c) 2007-2020 the FFmpeg developers
Input #0, matroska,webm, from 'The Apprentice (UK) S10E01.mkv':
  Metadata:
    ENCODER         : Stupid Ocelot
  Duration: 00:58:26.22, start: 0.080000, bitrate: 2178 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
    Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s (default)

I provide a cut down 2 minute clip of an hour long licensed episode which reproduces it, but I'm not sure about the repo's policy on copyrighted material? Unfortunately all content that exhibits this behaviour (so far) is privately purchased media.

Additional Info

ExoPlayer version number: 2.16.1
Android version: 10
Android device: Sony Bravia 4K VH2 (A80J)

I've tried the additional steps to eliminate external factors:

Completely disabling FFMPEG extensions with EXTENSION_RENDERER_MODE_OFF
Confirmed the content isn't at fault by playing at a faster speed with libVLC
Had this fail with ac3 and dts audio codecs (eliminating a single codec as the point of failure)

icbaker commented 2 years ago

Have you tried reproducing on any other devices (e.g. a phone) to see if this is specific to the Sony Bravia TV you're using?

Please provide content we can use to try and reproduce the problem, and please provide the result of adb bugreport after reproducing the error in the ExoPlayer demo app.

The exception message (arguments out of range) is defined in android_media_AudioTrack_set_playback_params and it will be attached to any non-OK response from AudioTrack::setPlaybackRate. It looks like there's additional warning logs in the AudioTrack method that will hopefully be visible in a bug report and may give a better clue as to exactly what property of the playback speed is causing a problem on this device.

As an aside: We've seen serious 'frame drop' symptoms when trying to play content at non-1.0x speed on Sony Bravia TVs (https://github.com/google/ExoPlayer/issues/7683). I don't know if the model you're using is affected by this issue. One wild guess is that the TV is deliberately not accepting the speed adjustment because it knows it will result in a bad experience (as I say, no concrete evidence that this is true).

DavidFair commented 2 years ago

Have you tried reproducing on any other devices (e.g. a phone) to see if this is specific to the Sony Bravia TV you're using?

So I've reproduced it on a completely different make and model of TV (hence the delay): Android version: 9 Android device: JVC LT-40CA890

Please provide content we can use to try and reproduce the problem, and please provide the result of adb bugreport after reproducing the error in the ExoPlayer demo app.

Content: https://drive.google.com/file/d/1B67k_9rDGUIBgNPdFp2vSlJ_hiRMsfJC/view?usp=sharing (I'm simply hosting with an Nginx docker instance locally: docker run -p 8080:80 -v /tmp/repro.mkv:/usr/share/nginx/html/repro.mkv:ro nginx && adb shell am start -a com.google.android.exoplayer.demo.action.VIEW -d http://192.168.10.31:8080/repro.mkv )

I can reproduce this on physical devices using the demo app, but not the emulator where it will always respect playback speed.

The exception message (arguments out of range) is defined in android_media_AudioTrack_set_playback_params and it will be attached to any non-OK response from AudioTrack::setPlaybackRate. It looks like there's additional warning logs in the AudioTrack method that will hopefully be visible in a bug report and may give a better clue as to exactly what property of the playback speed is causing a problem on this device.

Interestingly the emulator reports the audio track is not supported, but the physical device plays it fine, so I feel this confirms the audio track as the culprit: bugreport-BRAVIA_VH2_EU-QTG3.200305.006.S118-2022-03-21-20-51-37.zip

I'm happy to do any other testing you need too.

As an aside: We've seen serious 'frame drop' symptoms when trying to play content at non-1.0x speed on Sony Bravia TVs (#7683). I don't know if the model you're using is affected by this issue. One wild guess is that the TV is deliberately not accepting the speed adjustment because it knows it will result in a bad experience (as I say, no concrete evidence that this is true).

I originally thought this was true, hence I grabbed a non-Sony TV to confirm it too. Good shout though for me to test it

google / ExoPlayer