search

androidx / media

Jetpack Media3 support libraries for media use cases, including ExoPlayer, an extensible media player for Android
https://developer.android.com/media/media3
Apache License 2.0
1.59k stars 377 forks source link

Calling `setVideoSurfaceView` freezes playback and prints Playback error #457

Open rigor789 opened 1 year ago

rigor789 commented 1 year ago

Media3 Version

Media3 1.1.0-alpha01

Devices that reproduce the issue

Devices that do not reproduce the issue

Reproducible in the demo app?

Yes

Reproduction steps

In the demo-surface apply the following diff:

diff --git a/demos/surface/build.gradle b/demos/surface/build.gradle
index 09bb045308..903553f423 100644
--- a/demos/surface/build.gradle
+++ b/demos/surface/build.gradle
@@ -25,7 +25,7 @@ android {
     defaultConfig {
         versionName project.ext.releaseVersion
         versionCode project.ext.releaseVersionCode
-        minSdkVersion 29
+        minSdkVersion 28
         targetSdkVersion project.ext.appTargetSdkVersion
     }

diff --git a/demos/surface/src/main/java/androidx/media3/demo/surface/MainActivity.java b/demos/surface/src/main/java/androidx/media3/demo/surface/MainActivity.java
index bb3d20c094..12ab807796 100644
--- a/demos/surface/src/main/java/androidx/media3/demo/surface/MainActivity.java
+++ b/demos/surface/src/main/java/androidx/media3/demo/surface/MainActivity.java
@@ -162,10 +162,10 @@ public final class MainActivity extends Activity {
   public void onDestroy() {
     super.onDestroy();
     if (isOwner && isFinishing()) {
-      if (surfaceControl != null) {
-        surfaceControl.release();
-        surfaceControl = null;
-      }
+//      if (surfaceControl != null) {
+//        surfaceControl.release();
+//        surfaceControl = null;
+//      }
       if (videoSurface != null) {
         videoSurface.release();
         videoSurface = null;
@@ -227,13 +227,14 @@ public final class MainActivity extends Activity {
     player.play();
     player.setRepeatMode(Player.REPEAT_MODE_ALL);

-    surfaceControl =
-        new SurfaceControl.Builder()
-            .setName(SURFACE_CONTROL_NAME)
-            .setBufferSize(/* width= */ 0, /* height= */ 0)
-            .build();
-    videoSurface = new Surface(surfaceControl);
-    player.setVideoSurface(videoSurface);
+//    surfaceControl =
+//        new SurfaceControl.Builder()
+//            .setName(SURFACE_CONTROL_NAME)
+//            .setBufferSize(/* width= */ 0, /* height= */ 0)
+//            .build();
+//    videoSurface = new Surface(surfaceControl);
+//    player.setVideoSurface(videoSurface);
+
     MainActivity.player = player;
   }

@@ -266,20 +267,26 @@ public final class MainActivity extends Activity {
   }

   private static void reparent(@Nullable SurfaceView surfaceView) {
-    SurfaceControl surfaceControl = Assertions.checkNotNull(MainActivity.surfaceControl);
-    if (surfaceView == null) {
-      new SurfaceControl.Transaction()
-          .reparent(surfaceControl, /* newParent= */ null)
-          .setBufferSize(surfaceControl, /* w= */ 0, /* h= */ 0)
-          .setVisibility(surfaceControl, /* visible= */ false)
-          .apply();
+    if(surfaceView == null) {
+      player.clearVideoSurface();
     } else {
-      SurfaceControl newParentSurfaceControl = surfaceView.getSurfaceControl();
-      new SurfaceControl.Transaction()
-          .reparent(surfaceControl, newParentSurfaceControl)
-          .setBufferSize(surfaceControl, surfaceView.getWidth(), surfaceView.getHeight())
-          .setVisibility(surfaceControl, /* visible= */ true)
-          .apply();
+      player.clearVideoSurface();
+      player.setVideoSurfaceView(surfaceView);
     }
+//    SurfaceControl surfaceControl = Assertions.checkNotNull(MainActivity.surfaceControl);
+//    if (surfaceView == null) {
+//      new SurfaceControl.Transaction()
+//          .reparent(surfaceControl, /* newParent= */ null)
+//          .setBufferSize(surfaceControl, /* w= */ 0, /* h= */ 0)
+//          .setVisibility(surfaceControl, /* visible= */ false)
+//          .apply();
+//    } else {
+//      SurfaceControl newParentSurfaceControl = surfaceView.getSurfaceControl();
+//      new SurfaceControl.Transaction()
+//          .reparent(surfaceControl, newParentSurfaceControl)
+//          .setBufferSize(surfaceControl, surfaceView.getWidth(), surfaceView.getHeight())
+//          .setVisibility(surfaceControl, /* visible= */ true)
+//          .apply();
+//    }
   }
 }

For clarity, the updated reparent method implementation:

private static void reparent(@Nullable SurfaceView surfaceView) {
  if(surfaceView == null) {
    player.clearVideoSurface();
  } else {
    player.clearVideoSurface();
    player.setVideoSurfaceView(surfaceView);
  }
}

Expected result

The video to switch playing on a new surface (seamlessly?).

Actual result

On the generic Amlogic 2.0 device, the playback errors out:

2023-06-12 13:30:23.217  4593-4605  a3.demo.surfac          androidx.media3.demo.surface         I  Background concurrent copying GC freed 4736(320KB) AllocSpace objects, 0(0B) LOS objects, 24% free, 2MB/3MB, paused 1.850ms total 121.740ms
2023-06-12 13:30:33.306  4593-4701  SurfaceUtils            androidx.media3.demo.surface         D  connecting to surface 0x93dfb008, reason connectToSurface
2023-06-12 13:30:33.306  4593-4701  MediaCodec              androidx.media3.demo.surface         I  [OMX.amlogic.avc.decoder.awesome] setting surface generation to 4703234
2023-06-12 13:30:33.306  4593-4701  SurfaceUtils            androidx.media3.demo.surface         D  disconnecting from surface 0x93dfb008, reason connectToSurface(reconnect)
2023-06-12 13:30:33.306  4593-4701  SurfaceUtils            androidx.media3.demo.surface         D  connecting to surface 0x93dfb008, reason connectToSurface(reconnect)
2023-06-12 13:30:33.308  4593-4702  SurfaceUtils            androidx.media3.demo.surface         D  disconnecting from surface 0x93dfb008, reason setNativeWindowSizeFormatAndUsage
2023-06-12 13:30:33.308  4593-4702  SurfaceUtils            androidx.media3.demo.surface         D  connecting to surface 0x93dfb008, reason setNativeWindowSizeFormatAndUsage
2023-06-12 13:30:33.308  4593-4702  SurfaceUtils            androidx.media3.demo.surface         D  set up nativeWindow 0x93dfb008 for 1280x720, color 0x11, rotation 0, usage 0x402b00
2023-06-12 13:30:33.308  4593-4702  ACodec                  androidx.media3.demo.surface         W  cannot change usage from 0x402933 to 0x402b00
2023-06-12 13:30:33.325  4593-4634  ExoPlayerImplInternal   androidx.media3.demo.surface         E  Playback error
                                                                                                      androidx.media3.exoplayer.ExoPlaybackException: Unexpected runtime error
                                                                                                          at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:636)
                                                                                                          at android.os.Handler.dispatchMessage(Handler.java:102)
                                                                                                          at android.os.Looper.loop(Looper.java:193)
                                                                                                          at android.os.HandlerThread.run(HandlerThread.java:65)
                                                                                                      Caused by: java.lang.IllegalArgumentException
                                                                                                          at android.media.MediaCodec.native_setSurface(Native Method)
                                                                                                          at android.media.MediaCodec.setOutputSurface(MediaCodec.java:1979)
                                                                                                          at androidx.media3.exoplayer.mediacodec.SynchronousMediaCodecAdapter.setOutputSurface(SynchronousMediaCodecAdapter.java:191)
                                                                                                          at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.setOutputSurfaceV23(MediaCodecVideoRenderer.java:1503)
                                                                                                          at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.setOutput(MediaCodecVideoRenderer.java:697)
                                                                                                          at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.handleMessage(MediaCodecVideoRenderer.java:635)
                                                                                                          at androidx.media3.exoplayer.ExoPlayerImplInternal.deliverMessage(ExoPlayerImplInternal.java:1574)
                                                                                                          at androidx.media3.exoplayer.ExoPlayerImplInternal.sendMessageToTarget(ExoPlayerImplInternal.java:1538)
                                                                                                          at androidx.media3.exoplayer.ExoPlayerImplInternal.sendMessageInternal(ExoPlayerImplInternal.java:1513)
                                                                                                          at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:541)
                                                                                                          at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                                                          at android.os.Looper.loop(Looper.java:193) 
                                                                                                          at android.os.HandlerThread.run(HandlerThread.java:65) 
2023-06-12 13:30:33.767  4593-4701  SurfaceUtils            androidx.media3.demo.surface         D  disconnecting from surface 0x9410d008, reason disconnectFromSurface

https://github.com/androidx/media/assets/879060/c82b15c0-a633-4016-821d-ef7982400ee9

On the MiBox 4, it eventually starts playing on the new surface, but takes a while (~5 seconds).

https://github.com/androidx/media/assets/879060/5b476a3a-dbd7-4b2b-8f03-d702a8ae2ab6

Media

Default media for the surface demo / Not applicable.

Bug Report

rigor789 commented 1 year ago

Note: Adding franklin to the evaluateDeviceNeedsSetOutputSurfaceWorkaround makes it behave the same as the MiBox 4.

https://github.com/androidx/media/blob/2fc189d6a40f116bd54da69ab9a065219f6973e7/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/MediaCodecVideoRenderer.java#L1807-L1820