Slowmotion playback for 4k content on new Android TV Chromecast (codename Sabrina)

[kpandroid]

• Clear reproduction steps including observed and expected behavior compile demo project and run the test 60fps DASH -> 4K(MP4, H264, Clear)

• URI to test content for reproduction https://storage.googleapis.com/exoplayer-test-media-1/60fps/bbb-clear-2160/manifest.mpd

• ExoPlayer version number 2.13.0

• Android version Android TV 10

• Android device Chromecast

[kim-vde]

Could you explain more in detail what you observe or join a video showing how the content plays on the TV? A bug report would also be helpful. Thanks.

[kpandroid]

It's pretty easy - there are a lot of dropped video frames the parameter vfpo = -832000.

[kpandroid]

bugreport-sabrina_prod_stable-QTS2.200918.033-2021-02-08-15-17-55.zip

[kpandroid]

One more example that destroys the playback http://sylvan.apple.com/Aerials/2x/Videos/DB_D008_C010_4K_HDR_HEVC.mov

[christosts]

I tried with the 60fps H264 content. This video is encoded at 20Mbps and it looks like the device H264 decoder cannot decode fast enough this stream. The player logs indicate that too

MediaCodecVideoRenderer [
     Group:0, adaptive_supported=N/A [
       [X] Track:0, id=1, mimeType=video/avc, bitrate=19375046, codecs=avc1.640033, res=3840x2160, fps=60.0, supported=NO_EXCEEDS_CAPABILITIES
     ]
  ]

The player attempts to play the file although the decoder reports it's beyond its capabilities as a best effort - sometimes decoders can perform better than their reported capabilities. What appears as 'slow motion' is the combination of frames being dropped and the player trying to output video slightly slower to make the experience slightly better.

I also checked whether we can enable tunnelling on this device, and it looks like the hardware doesn't support tunnelling for H264, so I don't think we can do better for this specific content. I also tried some of the experimental rendering optimizations but they didn't really offer any improvement because the root cause of the problem is that this specific content is beyond the decoder's capabilities.

For the 2nd content (the mov file), mediainfo reports it's 4K, HEVC, 30fps at 8Mbps. I haven't tested yet on the Chromecast - I'll report back once I try it. I assume it has the same symptoms.

The Chromecast should support tunnelling for HEVC. To speed this investigation, would you be able to assist and try enabling tunnelling on the demo app and see it that addressed it? You can enable tunnelling calling setTunnelingEnabled(true) on the Parameters.Builder here.

[kpandroid]

@christosts setTunnelingEnabled(true) doesn't help at all - it seems it's DolbyVision HEVC and player just crashes with long error log. Generally my every 4K HEVC HDR test doesn't work. It's the same story again and again - huge amount of dropped frames. The most annoying thing during debugging is periodically WLAN disconnects (every 20 seconds!!!) - I understand that's not exoplayer problem...

[kpandroid]

one more small 4k hevc hdr example to test https://user-images.githubusercontent.com/56880093/107639207-db851100-6c70-11eb-8136-8ab86debf609.mp4

[christosts]

I've tried both the mov and mp4 files on a ChromeCast device. They both have issues, but different ones.

The mov file is HEVC 4K Dolby Vision encoded at 8Mbps. For this video, the platform decoder is failing to decode and MediaCodec is throwing an error when ExoPlayer is calling MediaCodec.dequeueOutputBuffer().

The mp4 file is HEVC 4K encoded at 23Mbps (according to mediainfo). This file is showing the same behavior as the file on the issue description: dropped frames and slower playback. Enabling tunnelling does improve the situation a bit, but the video still drops frames and overall I don't think it's watchable.

I tried playing all the mentioned videos on a TV and they play fine, so this seems to be down to the specific device and not an ExoPlayer problem.

For the cases playback drops frames, it appears content at so high bitrates (20Mbps+) is beyond the device's capabilities. I don't think the player can do anything here. Can you please try 4K content at a lower bitrate?

For the case of the Dolby Vision file, although this still looks like a device issue, I'll investigate further internally why the decoder fails.

[kpandroid]

@christosts according ffprobe the average bitrate of test.mp4 is about 7.4 Mbit.s - I'm sure it's pretty low bitrate for 4K HEVC HDR content. Moreover the problem exists for all hevc hdr videos, ever 1080p (with bitrate of few Mbit.s). Let me know if you need more examples. I'm pretty sure the the problem is device specific because the same hardware (amlogic line) is able to play all that videos on Xiaomi Mi Box (S) devices using > 100 Mbit.s content using the same version of Exoplayer. If you have the direct contact to Chromecast's guys please send them link to this issue.

[christosts]

@christosts according ffprobe the average bitrate of test.mp4 is about 7.4 Mbit.s

Yes, that was my mistake, I misread the mediainfo output.

There is a recent update on the Chromecast that improves 4K video. Can you please report the firmware of the device? https://support.google.com/chromecast/answer/7124014?hl=en&ref_topic=3447927

I noticed from the attached bug report that your device is on the recent Android build, but can you also report the firmware? There are instructions at "How to check your firmware version" on the link above.

[kpandroid]

@christosts I have the latest firmware (installed few days ago) - QTS2.200918.033.7115981

[christosts]

Can you please check the "Cast firmware version" as reported on the "How to check your firmware version" section? You should see something like 1.44.185164

[kpandroid]

The cast version is 1.50.228700

[christosts]

Thanks, my device has the same build and firmware.

So, now I'm facing the following: after a few cold reboots, the device started streaming the mov and mp4 files flawlessly. At the moment, I cannot get a repro of the failing decoder for the Dolby Vision case. I had performed several cold reboots before already and had tried different TVs (and more than one HDMI outputs on each TV), and the device was showing the problematic behavior but now it is working fine. I did not notice an update in between reboots. Not sure if this is unfortunate or a good sign? :)

1. Can you please try to cold reboot your chromecast too and give it another attempt?
2. If the dolby-vision video still has issues, can you please grab a bug-report after playing the dolby-vision video and attach it here?

[kpandroid]

@christosts

1. it doesn't help at all - few time cold rebooted, the same problems. About your assumption that the device can't handle 4k 60fps video - it should according the specs - https://store.google.com/us/product/chromecast_google_tv_specs?hl=en-US Resolution Up to 4K HDR, 60 FPS

2. logcat.txt bugreport-sabrina_prod_stable-QTS2.200918.033-2021-02-13-11-25-16.zip

[christosts]

Thank you for the new bug-report and logs, it is the same error I got with that video. I will forward them to the ChromeCast team.

1. About your assumption that the device can't handle 4k 60fps video - it should according the specs

This was my assumption from the first video, which was encoded with H264 at 60fps, 4K resolution at 20Mbps. I observed from the ExoPlayer perspective that the decoder did not decode fast enough. That does not mean the device cannot do 4K HDR 60fps in general, it could have been that the specific bitrate was too high for the H264 decoder. It may very well be the device can achieve much better performance with other decoders (HECV or VP9).

The other two videos: after several reboots, my device started playing them so it's definitely capable of playing them. I'll forward the bug-reports internally to see what may be causing this.

[kpandroid]

@christosts I've ordered one more Chromecast Sabrina in order to check the idea that the current is just defect (I've also permanently wlan connection drops every 30 seconds). I'll let you know if it helps.

[christosts]

for reference [internal ref b/180720722]

[kpandroid]

@christosts got one more Chromecast to test - the same picture.

[christosts]

I'm reporting from the Chromecast team:

Issue cannot be reproduced at the moment. The bugreport that you attached had a few network errors while playing the content. The Chromecast team asked if it's possible to play the content locally from your device to eliminate the possibility of the error being related from the network errors. If the issue persists, please attach a new bugreport.

Thanks

[google-oss-bot]

Hey @kpandroid. We need more information to resolve this issue but there hasn't been an update in 14 weekdays. I'm marking the issue as stale and if there are no new updates in the next 7 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[kpandroid]

@christosts just checked the example https://user-images.githubusercontent.com/56880093/107639207-db851100-6c70-11eb-8136-8ab86debf609.mp4 and new ExoPlayer 2.13.2. See the attached logcat please: logcat.txt

most interesting lines are:

2021-03-15 16:19:40.544 26707-27062/com.google.android.exoplayer2.demo D/CCodecConfig: config failed => CORRUPTED ... 2021-03-15 16:19:40.546 26707-27062/com.google.android.exoplayer2.demo D/CCodec: client requested max input size 1181, which is smaller than what component recommended (8192); overriding with component recommendation. 2021-03-15 16:19:40.546 26707-27062/com.google.android.exoplayer2.demo W/CCodec: This behavior is subject to change. It is recommended that app developers double check whether the requested max input size is in reasonable range.

[kpandroid]

Here it the bugreport for moved the remote mp4 test https://user-images.githubusercontent.com/56880093/107639207-db851100-6c70-11eb-8136-8ab86debf609.mp4 to "asset:///test.mp4

bugreport-sabrina_prod_stable-QTS2.200918.033-2021-03-15-17-39-44.zip

[christosts]

Thank you @kpandroid. If it's not too much trouble, can you repeat the same and play locally the 4K dolby vision video? http://sylvan.apple.com/Aerials/2x/Videos/DB_D008_C010_4K_HDR_HEVC.mov

If it still crashes, can you upload a bug-report too

[google-oss-bot]

Hey @kpandroid. We need more information to resolve this issue but there hasn't been an update in 14 weekdays. I'm marking the issue as stale and if there are no new updates in the next 7 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@kpandroid if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.