Noticalbe gaps between h264 video file playback

[StaticBR]

Issue description

I'm trying to play several h264 videos seamlessly in a playlist. Unfortunately some of our files produce noticable gaps (stutter) between the video files other video files don't. Currently I'm not albe to pinpoint the error, all have been created with the same converter settings. We have currently the presumption that this maybe has something to do with the length of the videos.

Reproduction steps

Prerequisites: ExoPlayer Demo App & Sample Videos from sample_videos.zip Upload the two sample videos from sample_videos.zip to an Web Server. Add a new playlist to the media.exolist.json, containing the two URLs of the sample files. You will see stutter / gap between the two videos.

Link to test content

sample_videos.zip This videos only contain a h264 video track. I removed tha audio track via ffmpeg to ensure that the gapless playback will not be caused by an error within the audio track gapless playback metadata. The gaps happen with and without the audio track.

Version of ExoPlayer being used

r2.8.2 (current master)

Device(s) and version(s) of Android being used

LGE Nexus 5 Android 7.1.2 ( LineageOS )

A full bug report captured from the device

If this is needed, please give me a short notice, and I send it via Email.

[StaticBR]

@botaydotcom Did you allready had the time to look into this issue? Is there anything I can do to support you?

[botaydotcom]

I think I can reproduce the issue on certain devices. Will try that see if I can find out the root cause.

[StaticBR]

@botaydotcom are there any news on this issue?

[botaydotcom]

Unfortunately I haven't got the time to continue working on this yet. I'll spend time on this issue later this week.

[StaticBR]

@botaydotcom die you habe time to look into this issue last week?

[botaydotcom]

Looked a bit into this, and I still couldn't find any problem here. I tested using a Nexus 6 (7.1.2) vs a Pixel 2 XL (8.1.0). In both cases, after the first video, the media codec is released and re-inited, and somehow on the Nexus 6, this leads to a brief (~0.5s) freezes, while on the Pixel 2 XL, this is mostly smooth (I still can find a very brief freeze though). I don't know if this is the limitation from the framework when release-reacquire media codec, or if there is some other issues on the older device (and older platform) here. Reassign to @ojw28 to look further into this.

[ojw28]

The problem is that the MediaCodec is being released and re-initialized, which can be slow. In some cases this is unavoidable, however in the example provided here we really should be able to avoid it and ensure the transition is nice as smooth.

The reason we're re-initializing in this case is that format.maxInputSize is set for both items, and the second item has a larger max input size (first item: 24K, second item: 32K). We initially configure the MediaCodec with the exact max input size from the first item. Because MediaCodec doesn't allow changing this value, when we get to the second item we have to release and initialize a new MediaCodec with larger input buffers.

To avoid this problem, when we configure the MediaCodec, we should probably size the input buffers to be larger than the format requires by some factor (e.g. 2x). With the additional constraint that we should never exceed the theoretical maximum for the codec, given the format width and height. This would allow us to re-use the same MediaCodec unless the bit-rate changes fairly dramatically. @andrewlewis - Any thoughts on exactly how this could work.

@StaticBR - If you want to test this theory, try making a playlist containing (1_vid, 2_vid, 1_vid, 2_vid). You'll probably see the pause on the first transition, but not on the second or third (by which time the MediaCodec instance has input buffers large enough for both videos).

[StaticBR]

@ojw28 Is there a way I can force the format.maxInputSize is set higher when initalizing the MediaCodec? Maybe there is somthing i can change within die Video, to force the Exo Player to do this?

Can you pin me down the Source Code location where exactly this is happening? So I can have a closer look?

Thanks.

[ojw28]

It would probably work to override MediaCodecVideoRenderer.getCodecMaxValues to return a CodecMaxValues with a larger maxInputSize value, as an interim measure until we can do something on our side.

[andrewlewis]

Sizing the input buffers with min(format.maxInputSize * someFactor, estimatedMaxInputSize) sounds fine. I guess it would be nicer to scale up the max input size only when the source is a concatenation, but I can't see an easy way to signal this at the moment.

[ojw28]

This should be addressed for most cases in dev-v2, so please give that a try. The fix will be in the 2.9.0 release (for which we do not yet have a solid ETA).

[StaticBR]

@ojw28 Thank you very much. We will give it a try.