google / ExoPlayer

This project is deprecated and stale. The latest ExoPlayer code is available in https://github.com/androidx/media
https://developer.android.com/media/media3/exoplayer
Apache License 2.0
21.7k stars 6.02k forks source link

how is it custom Extractor? #7858

Closed zengcanxiang closed 4 years ago

zengcanxiang commented 4 years ago

[REQUIRED] Searched documentation and issues

https://exoplayer.dev/doc/

implementation 'com.google.android.exoplayer:exoplayer-core:2.11.8'

[REQUIRED] Question

We added some extra bytes to the front of the mp4 file and renamed it.

When playing locally, I don’t want to decrypt the original file to the local first. Therefore, we intend to solve this problem by customizing Extractor and DataSource.

In the custom Extractor, I reused Mp4Extractor for actual reading.

In the sniff method, I skipped the correct number of bytes. The sniff of Mp4Extractor returned true.

But there was an exception of releaseCodec.

// code

class NDFExtractor : Extractor {
    private val mp4Extractor by lazy {
        Mp4Extractor()
    }

    override fun release() {
        Log.e("TAG", "NDFExtractor release")
        mp4Extractor.release()
    }

    override fun init(output: ExtractorOutput?) {
        mp4Extractor.init(output)
    }

    override fun seek(position: Long, timeUs: Long) {
        mp4Extractor.seek(position, timeUs)
    }

    override fun sniff(input: ExtractorInput?): Boolean {
        Log.e("TAG", "1111 ${mp4Extractor.sniff(input)}")
        input?.skip(2)
        val result = mp4Extractor.sniff(input)
        Log.e("TAG", "1111 $result")
        return result
    }

    override fun read(input: ExtractorInput?, seekPosition: PositionHolder?): Int {
        var isResult = mp4Extractor.sniff(input)
        Log.e("TAG", "2222 $isResult")
        return mp4Extractor.read(input, seekPosition)
    }

}

A full bug report captured from the device

2020-09-06 11:41:48.553 14763-14933/ E/TAG: 1111 false
2020-09-06 11:41:48.553 14763-14933/ E/TAG: 1111 true
2020-09-06 11:41:48.553 14763-14933/ E/TAG: 2222 true

2020-09-06 17:11:19.403 23820-24005/ E/ACodec: signalError(omxError 0x80001001, internalError -2147483648)
2020-09-06 17:11:19.403 23820-24005/ E/ACodec: [OMX.qcom.video.decoder.avc] ERROR(0x80001000)
2020-09-06 17:11:19.403 23820-24005/ E/ACodec: signalError(omxError 0x80001000, internalError -2147483648)
2020-09-06 17:11:19.403 23820-24004/ E/MediaCodec: Codec reported err 0x80001001, actionCode 0, while in state 6
2020-09-06 17:11:19.403 23820-24005/ E/ACodec: signalError(omxError 0x80001001, internalError -2147483648)
2020-09-06 17:11:19.405 23820-24004/ E/MediaCodec: Codec reported err 0x80001000, actionCode 0, while in state 0
2020-09-06 17:11:19.405 23820-24004/ E/MediaCodec: Codec reported err 0x80001001, actionCode 0, while in state 0
 E/ExoPlayerImplInternal: Renderer error: index=0, type=video, format=Format(1, null, null, video/avc, null, -1, null, [2560, 1440, 60.0], [-1, -1]), rendererSupport=YES
      com.google.android.exoplayer2.ExoPlaybackException: java.lang.IllegalStateException
        at com.google.android.exoplayer2.BaseRenderer.createRendererException(BaseRenderer.java:359)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:735)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:224)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: java.lang.IllegalStateException
        at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
        at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2789)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:1520)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:721)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329) 
        at android.os.Handler.dispatchMessage(Handler.java:103) 
        at android.os.Looper.loop(Looper.java:224) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
2020-09-06 11:41:48.687 14763-14932/ E/ExoPlayerImplInternal: Disable failed.
      java.lang.IllegalStateException
        at android.media.MediaCodec.native_flush(Native Method)
        at android.media.MediaCodec.flush(MediaCodec.java:2194)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.flushOrReleaseCodec(MediaCodecRenderer.java:777)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.flushOrReleaseCodec(MediaCodecVideoRenderer.java:724)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onDisabled(MediaCodecRenderer.java:636)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onDisabled(MediaCodecVideoRenderer.java:569)
        at com.google.android.exoplayer2.BaseRenderer.disable(BaseRenderer.java:168)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.disableRenderer(ExoPlayerImplInternal.java:1142)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:891)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:224)
        at android.os.HandlerThread.run(HandlerThread.java:67)
2020-09-06 11:41:48.692 14763-14939/ E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75fb535fb0
2020-09-06 11:41:48.692 14763-14939/ E/Surface: getSlotFromBufferLocked: unknown buffer: 0x75fb536020
2020-09-06 11:41:48.698 14763-14932/ E/ExoPlayerImplInternal: Reset failed.
      java.lang.IllegalStateException
        at android.media.MediaCodec.native_stop(Native Method)
        at android.media.MediaCodec.stop(MediaCodec.java:2147)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.releaseCodec(MediaCodecRenderer.java:671)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.releaseCodec(MediaCodecVideoRenderer.java:714)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onReset(MediaCodecRenderer.java:643)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onReset(MediaCodecVideoRenderer.java:578)
        at com.google.android.exoplayer2.BaseRenderer.reset(BaseRenderer.java:175)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:900)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:224)
        at android.os.HandlerThread.run(HandlerThread.java:67)
kim-vde commented 4 years ago

It is not clear to me why you would have a custom extractor.

You could instead have a custom data source that would wrap another data source and read from the second byte of the file. Something like:

public final class CustomDataSource implements DataSource {

  private final DefaultDataSource defaultDataSource;

  public CustomDataSource(Context context) {
    defaultDataSource = new DefaultDataSource(context, Util.getUserAgent(context, "myapp"), false);
  }

  @Override
  public void addTransferListener(TransferListener transferListener) {
    defaultDataSource.addTransferListener(transferListener);
  }

  @Override
  public long open(DataSpec dataSpec) throws IOException {
    DataSpec newDataSpec = dataSpec.buildUpon().setPosition(dataSpec.position + 2).build();
    return defaultDataSource.open(newDataSpec);
  }

  @Nullable
  @Override
  public Uri getUri() {
    return defaultDataSource.getUri();
  }

  @Override
  public Map<String, List<String>> getResponseHeaders() {
    return defaultDataSource.getResponseHeaders();
  }

  @Override
  public void close() throws IOException {
    defaultDataSource.close();
  }

  @Override
  public int read(byte[] target, int offset, int length) throws IOException {
    return defaultDataSource.read(target, offset, length);
  }
}

Would that work?

zengcanxiang commented 4 years ago

@kim-vde This code is ok on some phones. However, the same video still throws MediaCodec error on some mobile phones. I thought it was a problem with the original video file, but when I played mp4, there was no problem.

phone type is mix2s ,

 I/MediaCodec: CreateByComponentName(), name = OMX.qcom.video.decoder.avc
2020-09-09 11:44:37.081 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: new MediaCodec()
2020-09-09 11:44:37.082 19472-19596/com.mihoyo.videowallpaper I/ACodec: new ACodec()
2020-09-09 11:44:37.082 19472-19596/com.mihoyo.videowallpaper D/ACodec: ACodec() multimedia.audio.effect.type = dirac
2020-09-09 11:44:37.085 19472-19603/com.mihoyo.videowallpaper I/ACodec: onAllocateComponent()
2020-09-09 11:44:37.087 19472-19603/com.mihoyo.videowallpaper I/OMXClient: IOmx service obtained
2020-09-09 11:44:37.135 19472-19603/com.mihoyo.videowallpaper I/ACodec: Successfully allocate component [OMX.qcom.video.decoder.avc]
2020-09-09 11:44:37.138 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: configure()
2020-09-09 11:44:37.139 19472-19602/com.mihoyo.videowallpaper D/SurfaceUtils: connecting to surface 0x737dea8010, reason connectToSurface
2020-09-09 11:44:37.139 19472-19602/com.mihoyo.videowallpaper I/MediaCodec: [OMX.qcom.video.decoder.avc] setting surface generation to 19939329
2020-09-09 11:44:37.139 19472-19602/com.mihoyo.videowallpaper D/SurfaceUtils: disconnecting from surface 0x737dea8010, reason connectToSurface(reconnect)
2020-09-09 11:44:37.140 19472-19602/com.mihoyo.videowallpaper D/SurfaceUtils: connecting to surface 0x737dea8010, reason connectToSurface(reconnect)
2020-09-09 11:44:37.141 19472-19603/com.mihoyo.videowallpaper I/ACodec: configureCodec() mime = video/avc
2020-09-09 11:44:37.142 19472-19603/com.mihoyo.videowallpaper I/ExtendedACodec: setupVideoDecoder()
2020-09-09 11:44:37.144 19472-19603/com.mihoyo.videowallpaper I/ExtendedACodec: Decoder will be in frame by frame mode
2020-09-09 11:44:37.150 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: Video start()
2020-09-09 11:44:37.169 19472-19603/com.mihoyo.videowallpaper D/SurfaceUtils: set up nativeWindow 0x737dea8010 for 2560x1440, color 0x7fa30c06, rotation 0, usage 0x20002900
2020-09-09 11:44:37.183 19472-19603/com.mihoyo.videowallpaper E/ACodec: signalError(omxError 0x80001001, internalError -2147483648)
2020-09-09 11:44:37.183 19472-19603/com.mihoyo.videowallpaper D/IAtlas: IAtlas::init CallingPid 19472
2020-09-09 11:44:37.183 19472-19603/com.mihoyo.videowallpaper D/IAtlas: IAtlas::init this 0x737daf1260
2020-09-09 11:44:37.184 19472-19602/com.mihoyo.videowallpaper E/MediaCodec: Codec reported err 0x80001001, actionCode 0, while in state 6
2020-09-09 11:44:37.184 19472-19602/com.mihoyo.videowallpaper D/SurfaceUtils: disconnecting from surface 0x737dea8010, reason disconnectFromSurface
2020-09-09 11:44:37.184 19472-19603/com.mihoyo.videowallpaper E/ACodec: [OMX.qcom.video.decoder.avc] ERROR(0x80001000)
2020-09-09 11:44:37.184 19472-19603/com.mihoyo.videowallpaper E/ACodec: signalError(omxError 0x80001000, internalError -2147483648)
2020-09-09 11:44:37.184 19472-19603/com.mihoyo.videowallpaper D/IAtlas: IAtlas::init CallingPid 19472
2020-09-09 11:44:37.185 19472-19603/com.mihoyo.videowallpaper D/IAtlas: IAtlas::init this 0x737daf1260
2020-09-09 11:44:37.186 19472-19602/com.mihoyo.videowallpaper E/MediaCodec: Codec reported err 0x80001000, actionCode 0, while in state 0
2020-09-09 11:44:37.192 19472-19596/com.mihoyo.videowallpaper E/ExoPlayerImplInternal: Renderer error: index=0, type=video, format=Format(1, null, null, video/avc, null, -1, null, [2560, 1440, 60.0], [-1, -1]), rendererSupport=YES
      com.google.android.exoplayer2.ExoPlaybackException: java.lang.IllegalStateException
        at com.google.android.exoplayer2.BaseRenderer.createRendererException(BaseRenderer.java:359)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:735)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:238)
        at android.os.HandlerThread.run(HandlerThread.java:67)
     Caused by: java.lang.IllegalStateException
        at android.media.MediaCodec.native_dequeueOutputBuffer(Native Method)
        at android.media.MediaCodec.dequeueOutputBuffer(MediaCodec.java:2826)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:1520)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:721)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:599) 
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:329) 
        at android.os.Handler.dispatchMessage(Handler.java:103) 
        at android.os.Looper.loop(Looper.java:238) 
        at android.os.HandlerThread.run(HandlerThread.java:67) 
2020-09-09 11:44:37.193 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: Video flush()
2020-09-09 11:44:37.195 19472-19596/com.mihoyo.videowallpaper E/ExoPlayerImplInternal: Disable failed.
      java.lang.IllegalStateException
        at android.media.MediaCodec.native_flush(Native Method)
        at android.media.MediaCodec.flush(MediaCodec.java:2231)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.flushOrReleaseCodec(MediaCodecRenderer.java:777)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.flushOrReleaseCodec(MediaCodecVideoRenderer.java:724)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onDisabled(MediaCodecRenderer.java:636)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onDisabled(MediaCodecVideoRenderer.java:569)
        at com.google.android.exoplayer2.BaseRenderer.disable(BaseRenderer.java:168)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.disableRenderer(ExoPlayerImplInternal.java:1142)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:891)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:238)
        at android.os.HandlerThread.run(HandlerThread.java:67)
2020-09-09 11:44:37.196 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: stop()
2020-09-09 11:44:37.199 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: Video release()
2020-09-09 11:44:37.204 19472-19603/com.mihoyo.videowallpaper E/Surface: getSlotFromBufferLocked: unknown buffer: 0x737daf12d0
2020-09-09 11:44:37.204 19472-19603/com.mihoyo.videowallpaper W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 20 to native window
2020-09-09 11:44:37.205 19472-19603/com.mihoyo.videowallpaper E/Surface: getSlotFromBufferLocked: unknown buffer: 0x737daf1340
2020-09-09 11:44:37.206 19472-19603/com.mihoyo.videowallpaper W/ACodec: [OMX.qcom.video.decoder.avc] can not return buffer 19 to native window
2020-09-09 11:44:37.215 19472-19596/com.mihoyo.videowallpaper I/MediaCodec: ~MediaCodec()
2020-09-09 11:44:37.215 19472-19596/com.mihoyo.videowallpaper I/ACodec: ~ACodec()
2020-09-09 11:44:37.217 19472-19596/com.mihoyo.videowallpaper E/ExoPlayerImplInternal: Reset failed.
      java.lang.IllegalStateException
        at android.media.MediaCodec.native_stop(Native Method)
        at android.media.MediaCodec.stop(MediaCodec.java:2184)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.releaseCodec(MediaCodecRenderer.java:671)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.releaseCodec(MediaCodecVideoRenderer.java:714)
        at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.onReset(MediaCodecRenderer.java:643)
        at com.google.android.exoplayer2.video.MediaCodecVideoRenderer.onReset(MediaCodecVideoRenderer.java:578)
        at com.google.android.exoplayer2.BaseRenderer.reset(BaseRenderer.java:175)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:900)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:850)
        at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:382)
        at android.os.Handler.dispatchMessage(Handler.java:103)
        at android.os.Looper.loop(Looper.java:238)
        at android.os.HandlerThread.run(HandlerThread.java:67)
zengcanxiang commented 4 years ago

@kim-vde this is file 1596375533958_30.zip

zengcanxiang commented 4 years ago

@kim-vde At the beginning, inputStream skipped bytes when I used it. The same is true. It can be rendered on some mobile phones. Will it be exoplayer some attributes that require model compatibility, let me open it?

context.contentResolver.openInputStream(mediaUri)

SimpleExoPlayer.Builder(context)
            .build().also {
                it.videoScalingMode = C.VIDEO_SCALING_MODE_SCALE_TO_FIT_WITH_CROPPING
                it.addListener(listener)
            }
kim-vde commented 4 years ago

I am not sure I understand. Is the issue you are having about the 2 bytes you added to the file, or is it about the original file (without the added bytes)? I tried to play the original file (i.e. the file that you sent but without the 2 first bytes) on my phone and it works well. Or are you saying the original file does not play on some devices but plays on others?

Looking at the stacktrace, the issue seems to be about a codec-related resource being unavailable (see similar issue #5173).

Something that could help figuring out what is happening is to play the original file on a device that has the issue with the demo app, and to add the bug report on this issue.