HEVC adaptation issues when switching profiles

[metalg]

i'm experiencing an issue in a device which reference board is the qualcomm sdm845 and confirmed the issue is reproducible in a Samsung s9 phone, both have the same hevc video decoder OMX.qcom.video.decoder.hevc and the secure version.

this is the log of the decoders.

02-12 13:24:16.872 25185 25185 D EventLogger: downstreamFormat [eventTime=0.12, mediaPos=0.00, window=0, period=0, id=1, mimeType=video/hevc, codecs=hvc1.2.4.L150.A0, res=3840x2160]
02-12 13:24:16.916 25185 25185 D EventLogger: videoDecoderInitialized [eventTime=0.17, mediaPos=0.00, window=0, period=0, OMX.qcom.video.decoder.hevc]
02-12 13:24:16.917 25185 25185 D EventLogger: videoInputFormat [eventTime=0.17, mediaPos=0.00, window=0, period=0, id=1, mimeType=video/hevc, codecs=hvc1.2.4.L150.A0, res=3840x2160]
02-12 13:24:36.085 25185 25185 D EventLogger: downstreamFormat [eventTime=19.33, mediaPos=0.00, window=1, period=1, id=1, mimeType=video/hevc, codecs=hvc1.1.6.L150.90, res=3840x2160]
02-12 13:24:36.086 25185 25185 D EventLogger: videoInputFormat [eventTime=19.34, mediaPos=0.00, window=1, period=1, id=1, mimeType=video/hevc, codecs=hvc1.1.6.L150.90, res=3840x2160]

when playing the videos in this order in a playlist order: hvc1.2.4.L150.A0, hvc1.1.6.L150.90

Decoder is configured for the first file (hvc1.2.4.L150.A0 codec profile)and It plays fine. But after app started to decode/play the next video file (which codec profile is hvc1.1.6.L150.90) - no error reported from decoder, app still uses decoder configured for hvc1.2.4.L150.A0 and we observe video distortion.

if playing the videos in the reverse order both the videos play fine.

if using a pixel device, issue was not observed. however, the decoder named in the playback are different. the pixel uses c2.qti.hevc.decoder. It also does not report any errors. but the decoder initialization happens once only and works fine in both cases.

since my hardware is custom i've discarted a change in workaround code in mediacodecinfo.java https://github.com/google/ExoPlayer/blob/f1b37bc547e7194354fe31e240afa8a8fb0b5c6c/library/core/src/main/java/com/google/android/exoplayer2/mediacodec/MediaCodecInfo.java#L797 but may want to consider for off the shelf device such as the samsung s9

but for my immediate need there seems to be a workaround i can do by customizing the renderer factory/and video renderer in my app. reference https://github.com/metalg/testplayer/blob/master/app/src/main/java/com/example/concatenatingplayer/PlayListMediaCodecVideoRenderer.java#L38

is this the right approach to solve this issue i face?

[ojw28]

is this the right approach to solve this issue i face?

It's definitely an approach to workaround the issue you face. Whether it's the optimal approach is tricky to say, but it should be noted that targeting the workaround based on codec name does not ensure that you only target a single codec "implementation". In practice SoC vendors will use the same name for many different implementations or variants of their codec, and it's likely that only specific subset are actually affected.

If your app is only going to run on custom hardware then you generally don't need to worry about this. If it's not, then you might want to try targeting the workaround also at specific devices. There are some examples of doing this in the ExoPlayer codebase here. If you're not sure which devices are affected, and if there's no significant negative effect of enabling the workaround unnecessarily, then enabling in broadly (e.g., based only on codec name) might be fine. If there is significant negative effect, then targeting it more tightly can be beneficial, but working out how to target it becomes harder.

We will try and reproduce with the sample files provided (thanks!), see whether it's worthwhile adding a workaround directly into the ExoPlayer codebase, and if so spend a bit of time thinking about how we think it would be best targeted.

[ojw28]

I tested this on an Exynos based Galaxy S20 and saw strange behavior there as well (OMX.Exynos.hevc.dec). Playback continued without error, but after the transition the video was squashed into only the left side of the output surface, and the right side was solid green. On an older Exynos based Galaxy S8, the transition worked correctly.

[metalg]

The video distortion, i mentioned was two behaviors, one was the green on top of the video (i could literally see the video through the green), and the other was a comple scramble of the video. I'll post some screen shots of those when I get back.

On Sun, Mar 13, 2022, 5:09 PM Oliver Woodman @.***> wrote:

I tested this on an Exynos based Galaxy S20 and saw strange behavior there as well (OMX.Exynos.hevc.dec). Playback continued without error, but after the transition the video was squashed into only the left side of the output surface, and the right side was solid green. On an older Exynos based Galaxy S8, the transition worked correctly.

— Reply to this email directly, view it on GitHub https://github.com/google/ExoPlayer/issues/10052#issuecomment-1066214440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHM3XQEPE57P2EBYSWMBM3U7Z7TJANCNFSM5QOO33CQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[metalg]

Video distortion on the second video.

On Sun, Mar 13, 2022, 7:16 PM German Li @.***> wrote:

The video distortion, i mentioned was two behaviors, one was the green on top of the video (i could literally see the video through the green), and the other was a comple scramble of the video. I'll post some screen shots of those when I get back.

On Sun, Mar 13, 2022, 5:09 PM Oliver Woodman @.***> wrote:

I tested this on an Exynos based Galaxy S20 and saw strange behavior there as well (OMX.Exynos.hevc.dec). Playback continued without error, but after the transition the video was squashed into only the left side of the output surface, and the right side was solid green. On an older Exynos based Galaxy S8, the transition worked correctly.

— Reply to this email directly, view it on GitHub https://github.com/google/ExoPlayer/issues/10052#issuecomment-1066214440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHM3XQEPE57P2EBYSWMBM3U7Z7TJANCNFSM5QOO33CQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[ojw28]

I'm not sure if your response above was attempting to attach some screenshots, but if it was, it didn't work. Please respond in the GitHub UI, rather than email, in order to ensure that any attachments are indeed attached correctly.

[metalg]

sorry i didn't realize that the email attachment was not uploaded.