Significant frame drop while playing 1920x1080 HLS video tracks on AFTSSS and AFTSS

[Koster35]

[REQUIRED] Issue description

Fire TV Stick - 3rd Gen (2020) (AFTSSS) and Fire TV Stick Lite - 1st Gen (2020) (AFTSS) consistently drop frames while playing 1920x1080 video tracks in HLS streams.

This issue started when we upgraded our ExoPlayer Amazon port version from v2.10.6 to v2.11.3. The issue also occurs on v2.12.1.

After some debugging, I have some insight as to why this started in v2.11: In both v2.10 and v2.11, ExoPlayer's MediaCodecVideoRenderer.getDecoderInfos() method call returns a list of decoders available on the device. On v2.10, the order of the codecs was always returned in the following order, on both of these Fire Stick devices: [OMX.MTK.VIDEO.DECODER.AVC, OMX.google.h264.decoder]

So, to decode our HLS stream, the ExoPlayer would choose the first decoder in the list, OMX.MTK.VIDEO.DECODER.AVC (the hardware decoder). When the ExoPlayer checks the Format of our 1920x1080 video track in the HLS stream against this hardware codec, it returns that the hardware codec does not support the 1920x1080 video track. So, it would never attempt to play the 1920x1080 video track.

Starting in ExoPlayer v2.11, the ExoPlayer's MediaCodecVideoRenderer.getDecoderInfos() implementation changed. This method started sorting the list of decoders by 'Format' support.

The sorting recognizes that the hardware decoder (OMX.MTK.VIDEO.DECODER.AVC) does not support the 1920x1080 video track in our HLS stream, but the software decoder (OMX.google.h264.decoder) does, so the sorting method call sorts the list of available decoders into the following order: [OMX.google.h264.decoder, OMX.MTK.VIDEO.DECODER.AVC]. The ExoPlayer then chooses the first decoder in the list, the software decoder, to decode the 1920x1080 video track. However, during playback, there is nearly constant frame drop causing a bad experience for the user.

I don't know what's causing the software codec to perform so poorly, but there are logs indicating frames being dropped:

2021-05-05 16:25:59.404 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.436 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 2, PTS = 800000 2021-05-05 16:25:59.478 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 3, PTS = 840000 2021-05-05 16:25:59.486 7423-7439/? I/exoplayer2.dem: Background concurrent copying GC freed 12043(757KB) AllocSpace objects, 4(272KB) LOS objects, 43% free, 7MB/13MB, paused 897us total 147.805ms 2021-05-05 16:25:59.521 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.571 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.584 513-601/? W/BestClock: java.time.DateTimeException: Missing NTP fix 2021-05-05 16:25:59.592 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 6, PTS = 960000 2021-05-05 16:25:59.602 315-496/? D/AudioFlinger: mixer(0xa5183780) throttle end: throttle time(3) 2021-05-05 16:25:59.638 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.644 513-601/? W/BestClock: java.time.DateTimeException: Missing NTP fix 2021-05-05 16:25:59.647 513-602/? W/BestClock: java.time.DateTimeException: Missing NTP fix 2021-05-05 16:25:59.655 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 7, PTS = 1000000 2021-05-05 16:25:59.704 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.708 315-496/? D/AudioFlinger: mixer(0xa5183780) throttle end: throttle time(2) 2021-05-05 16:25:59.745 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 1, PTS = 1080000 2021-05-05 16:25:59.785 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 2, PTS = 1120000 2021-05-05 16:25:59.836 315-496/? D/AudioFlinger: mixer(0xa5183780) throttle end: throttle time(3) 2021-05-05 16:25:59.838 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:25:59.879 315-496/? D/AudioFlinger: mixer(0xa5183780) throttle end: throttle time(3) 2021-05-05 16:25:59.888 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 4, PTS = 1200000 2021-05-05 16:25:59.919 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 5, PTS = 1240000 2021-05-05 16:25:59.971 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:26:00.032 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 7, PTS = 1320000 2021-05-05 16:26:00.072 319-319/? E/FrameEvents: updateAcquireFence: Did not find frame. 2021-05-05 16:26:00.082 513-601/? W/BestClock: java.time.DateTimeException: Missing NTP fix 2021-05-05 16:26:00.092 315-496/? D/AudioFlinger: mixer(0xa5183780) throttle end: throttle time(3) 2021-05-05 16:26:00.108 7423-7618/? I/CodecNameUnknown-MediaCodecVideoRenderer: dropOutputBuffer: bufferIndex = 1, PTS = 1400000

I also don't know why these two Fire Sticks are reporting that their hardware codec does not support the 1920x1080 video track. Seemingly, the same OMX.MTK.VIDEO.DECODER.AVC hardware codec can play 1920x1080 video tracks on the Fire TV Stick - 2nd Gen (2016-2019) (AFTT), but not on the newer models.

Finally, this may be related to #42 but I'm not sure.

[REQUIRED] Reproduction steps

This does not reproduce with the Apple HLS samples in the demo app. However, you can reproduce it in the demo app using this sample I found on Bitmovin's website: https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8

Notice the choppiness of the video and the logs being printed. If the issue does not reproduce the first time, try:

• Backing out of the video, then starting it again.
• Manually selecting the 1920x1080 video track.

[REQUIRED] Link to test content

https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8

I can provide our production HLS streams if more than this sample is needed.

[REQUIRED] A full bug report captured from the device

Fire TV Stick Lite - 1st Gen (2020) (AFTSS): bugreport-sheldon-PS7234-2021-05-05-16-39-26.zip

[REQUIRED] Version of ExoPlayer being used

v2.12.1 Amazon port v2.11.3 Amazon port

It seems to reproduce in all ExoPlayer versions (Amazon port or not) starting with v2.11.

[REQUIRED] Device(s) and version(s) of Android being used

• Fire TV Stick - 3rd Gen (2020) (AFTSSS)
• Fire TV Stick Lite - 1st Gen (2020) (AFTSS)

[Koster35]

I've also opened this issue on the ExoPlayer repo https://github.com/google/ExoPlayer/issues/9565

It appears the original bug report I submitted here may not have all the necessary info to debug the issue. If generating another bug report would be helpful, let me know.

[saumya-y]

Hi @Koster35, can you share the adb logcat with us and the FOS version running on your Fire TV Stick-3rd Gen(2020), Fire TV Stick Lite-1st Gen(2020) to help fast track this?

[Koster35]

Here is that info using the ExoPlayer Amazon port r2.13.3 demo app playing the same sample as above https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8

Device: Fire TV Stick Lite - 1st Gen (2020) (AFTSS) OS Version: Fire OS 7.2.4.2 Logcat: AFTSS_logcat.txt

Device: Fire TV Stick - 3rd Gen (2020) (AFTSSS) OS Version: Fire OS 7.2.4.2 Logcat: AFTSSS_logcat.txt

Thanks!

[gregni2]

We are seeing the same issue on many of our fire devices as well. Looking forward to seeing a resolution to this ticket. Thanks for addressing it!

[erocapbs]

I'm also seeing this issue when testing exoplayer. Hopefully there's a fix for this soon.

[beshenpbs]

As a result of this bug, we have had to downgrade the video quality served to all Amazon device users. A fast resolution to this issue would be greatly appreciated.

[wonderfulBee]

I am seeing the sane issue on exoplayer too. waiting for a solution```