Transformer Video Decoder error when using Overlays

jordond commented 10 months ago

Version

Media3 pre-release (alpha, beta or RC not in this list)

More version details

release-1.2.0-rc01

Devices that reproduce the issue

Samsung Z Flip 5, Android 13

Devices that do not reproduce the issue

Pixel 7 Pro, Android 13
Samsung Z 4 Fold - Android 13
Samsung S21 Ultra 5G - Android 12
Samsung S10e - Android 12
arm64 emulator - Android 13

Reproducible in the demo app?

Yes

Reproduction steps

Using the Transformer demo:

Launch demo, use the default preset file "720p H264 video and AAC audio"
Tap "Add video effects" and enable "Overlay logo & timer"
Hit the export button

Expected result

Transformer exports the video with the logo and timer overlayed

Actual result

Transformer throws a ExportException:

TransformerActivity      E  Export error
  androidx.media3.transformer.ExportException: VideoDecoder error: mediaFormat={exo-pixel-width-height-ratio-float=1.0, sar-width=1, is-forced-subtitle=0, allow-frame-drop=0, level=512, mime=video/avc, profile=8, codecs-string=avc1.64001F, csd-1=java.nio.HeapByteBuffer[pos=0 lim=10 cap=10], encoder-delay=0, sar-height=1, width=1280, is-autoselect=0, is-default=0, encoder-padding=0, frame-rate=29.97003, max-input-size=34677, rotation-degrees=0, height=720, csd-0=java.nio.HeapByteBuffer[pos=0 lim=28 cap=28]}, mediaCodecName=c2.qti.avc.decoder
     at androidx.media3.transformer.DefaultCodec.createExportException(DefaultCodec.java:411)
     at androidx.media3.transformer.DefaultCodec.createExportException(DefaultCodec.java:395)
     at androidx.media3.transformer.DefaultCodec.maybeDequeueInputBuffer(DefaultCodec.java:186)
     at androidx.media3.transformer.ExoAssetLoaderBaseRenderer.feedDecoderFromInput(ExoAssetLoaderBaseRenderer.java:286)
     at androidx.media3.transformer.ExoAssetLoaderBaseRenderer.render(ExoAssetLoaderBaseRenderer.java:113)
     at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1079)
     at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:529)
     at android.os.Handler.dispatchMessage(Handler.java:102)
     at android.os.Looper.loopOnce(Looper.java:226)
     at android.os.Looper.loop(Looper.java:313)
     at android.os.HandlerThread.run(HandlerThread.java:67)
 Caused by: java.lang.IllegalStateException
     at android.media.MediaCodec.native_dequeueInputBuffer(Native Method)
     at android.media.MediaCodec.dequeueInputBuffer(MediaCodec.java:2976)
     at androidx.media3.transformer.DefaultCodec.maybeDequeueInputBuffer(DefaultCodec.java:183)
     at androidx.media3.transformer.ExoAssetLoaderBaseRenderer.feedDecoderFromInput(ExoAssetLoaderBaseRenderer.java:286) 
     at androidx.media3.transformer.ExoAssetLoaderBaseRenderer.render(ExoAssetLoaderBaseRenderer.java:113) 
     at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1079) 
     at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:529) 
     at android.os.Handler.dispatchMessage(Handler.java:102) 
     at android.os.Looper.loopOnce(Looper.java:226) 
     at android.os.Looper.loop(Looper.java:313) 
     at android.os.HandlerThread.run(HandlerThread.java:67)

Media

Not working preset files:

720p H264 video and AAC audio
1080p H265 video and AAC audio
360p H264 video and AAC audio
360p VP8 video and Vorbis audio

All other presets seem to work just fine

Bug Report

[x] You will email the zip file produced by adb bugreport to android-media-github@google.com after filing this issue.

droid-girl commented 10 months ago

@jordond we have not received the bug report. Could you please share it again?

jordond commented 10 months ago

@jordond we have not received the bug report. Could you please share it again?

Just sent it again 👍

andrewlewis commented 10 months ago

Following up with the codec framework team on internal bug b/311206113.

Ktaewon commented 10 months ago

Is the exception I wrote below the same bug as this?

FATAL EXCEPTION: DefaultDispatcher-worker-8
    Process: com.kakao.openlink.sandbox.debug, PID: 15373
    androidx.media3.transformer.ExportException: VideoEncoder error: mediaFormat={exo-pixel-width-height-ratio-float=1.0, sar-width=1, color-format=2130708361, i-frame-interval=1.0, is-forced-subtitle=0, level=131072, mime=video/avc, profile=8, bitrate=2048000, priority=1, color-standard=2, operating-rate=2147483647, color-transfer=3, encoder-delay=0, sar-height=1, width=1280, bitrate-mode=1, is-autoselect=0, is-default=0, color-range=2, encoder-padding=0, frame-rate=30, rotation-degrees=0, height=1280}, mediaCodecName=c2.qti.avc.encoder
        at androidx.media3.transformer.DefaultCodec.createExportException(DefaultCodec.java:395)
        at androidx.media3.transformer.DefaultCodec.createExportException(DefaultCodec.java:379)
        at androidx.media3.transformer.DefaultCodec.releaseOutputBuffer(DefaultCodec.java:285)
        at androidx.media3.transformer.DefaultCodec.releaseOutputBuffer(DefaultCodec.java:265)
        at androidx.media3.transformer.VideoSamplePipeline$EncoderWrapper.releaseOutputBuffer(VideoSamplePipeline.java:570)
        at androidx.media3.transformer.VideoSamplePipeline.releaseMuxerInputBuffer(VideoSamplePipeline.java:310)
        at androidx.media3.transformer.SamplePipeline.feedMuxer(SamplePipeline.java:130)
        at androidx.media3.transformer.SamplePipeline.processData(SamplePipeline.java:67)
        at androidx.media3.transformer.TransformerInternal.drainPipelinesInternal(TransformerInternal.java:326)
        at androidx.media3.transformer.TransformerInternal.handleMessage(TransformerInternal.java:291)
        at androidx.media3.transformer.TransformerInternal.$r8$lambda$0Eq5y0bhoOvqh9wEU99UtqZE5oc(Unknown Source:0)
        at androidx.media3.transformer.TransformerInternal$$ExternalSyntheticLambda0.handleMessage(Unknown Source:2)
        at android.os.Handler.dispatchMessage(Handler.java:102)
        at android.os.Looper.loopOnce(Looper.java:226)
        at android.os.Looper.loop(Looper.java:313)
        at android.os.HandlerThread.run(HandlerThread.java:67)
        Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@45c7f21, Dispatchers.IO]
    Caused by: java.lang.IllegalStateException
        at android.media.MediaCodec.releaseOutputBuffer(Native Method)
        at android.media.MediaCodec.releaseOutputBufferInternal(MediaCodec.java:3689)
        at android.media.MediaCodec.releaseOutputBuffer(MediaCodec.java:3610)
        at androidx.media3.transformer.DefaultCodec.releaseOutputBuffer(DefaultCodec.java:281)
        ... 13 more

The device with the problem is samsung s22, android 13, one ui 5.1. It operates normally in samsung s21(android 14), google pixel 7(android 14), etc.

andrewlewis commented 10 months ago

That stack trace is too generic to be able to answer your question conclusively (it might be the same but it's probably different). When MediaCodec throws a runtime exception it's usually necessary to look at the logging before to identify the root cause of the problem.

Ktaewon commented 10 months ago

That stack trace is too generic to be able to answer your question conclusively (it might be the same but it's probably different). When MediaCodec throws a runtime exception it's usually necessary to look at the logging before to identify the root cause of the problem.

Thanks for reply. Then could you give me more information how to debug the root cause of the problem? Additionally, the jordond's issue is occured when i encode a video with videoEffects(Presentation.createForHeight(any value)) in Samsung s23(android 14).(Other devices below android 13 that i have are fine.) The issue is reproducible in the demo app.

andrewlewis commented 10 months ago

First step would be to get a full bug report produced via adb bugreport sent to android-media-github@google.com. But before doing that, please could you file a new issue (unless you are confident that it's the same problem as this issue)? It can be quite confusing if we end up discuss two different problems on the same issue.

Ktaewon commented 10 months ago

First step would be to get a full bug report produced via adb bugreport sent to android-media-github@google.com. But before doing that, please could you file a new issue (unless you are confident that it's the same problem as this issue)? It can be quite confusing if we end up discuss two different problems on the same issue.

I filed a new issue (https://github.com/androidx/media/issues/830). thanks.

andrewlewis commented 10 months ago

I verified the workaround on an S23+ device, which I think has the same chipset. I'll close this for now but please let us know if you continue to see a problem. Thanks!

Ktaewon commented 10 months ago

I verified the workaround on an S23+ device, which I think has the same chipset. I'll close this for now but please let us know if you continue to see a problem. Thanks!

I confirmed that there is no problem with my s23+! thank you! Can you tell me when the version left behind by this commit will be released?

andrewlewis commented 10 months ago

It will go into the next release (probably 1.2.1, or 1.3.0 otherwise) but I don't know when that will be.

androidx / media