AVPro Video Android Fast OES path causes black screen on Universal Render Pipeline (URP) with Bloom post processing enabled and HDR enabled

[tomeroberts91]

Describe the bug When on an Android device with Universal Render Pipeline being used, bloom on, HDR on, an Exo Player or Media Player, with Fast OES Path on, with the video being applied to a material with the Apply To Material component, the screen will go completely black on some Android devices, or very white/over-bloomed on others (seems to be Mali GPU vs Adreno GPU).

Notes: I debugged the application in RenderDoc and found that some of the pixels in the Video before bloom was applied had values of INF, which makes me think it's some sort of encoding/decoding or format error that's going on

Your Setup (please complete the following information):

• Unity version: 2019.3.15f1
• AVPro Video version: 1.11.5
• Operating system version: Android 10
• Device model: Samsung S10+ (SM-G975F) EU version
• Video specs (resolution, frame-rate, codec, file size): Using apple basic 16x9 stream from https://developer.apple.com/streaming/examples/ H.264 @30hz

To Reproduce

1. Open Example project (there will be an email sent after this has been written) a. Or try and set up a URP project with HDR turned on for the Quality Setting that deploys by default to Android (I believe it is Medium) with a video playing using Fast OES Path on a mesh/material
2. Deploy to Android Device
3. Run application
4. Toggle HDR on and off to see the black screen that happens while the video is playing (audio should still be heard in the background)

Logs If applicable, add error logs to help explain your problem. https://1drv.ms/u/s!Au8PukIbIUbEgr9RhEY94-iW5jOB-w

Screenshots If applicable, add screenshots to help explain your problem.

Videos If applicable, add a copy of your video or the URL https://1drv.ms/v/s!Au8PukIbIUbEgr9UKBU4chQpbGK3YA

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE

Instead email the link to us unitysupport@renderheads.com

[AndrewRH]

Have you tried enabling the option 'stop NaN propagation'?

[tomeroberts91]

Hi, thank you for the swift reply.

That does seem to fix the Bloom problem, not ideal as it adds a bit of a performance hit when enabled. On our actual project we had another issue when we did this that I forgot to include. The video moved around a lot when it should have been static on the mesh, like it was trying to apply the video to the wrong mesh.

Screenshot:

Video: https://1drv.ms/v/s!Au8PukIbIUbEgr9b4L0UbC2-ONp8Dw?e=auHgLe

This may be a seperate issue to do with static batching? as I can only get it to do it in the example project when I disable the static flag. Unfortunately as in our actual project the mesh is from an asset bundle, it doesn't have the static flag set (doesn't seem to come across in Unity, even if set in original bundle)

[AndrewRH]

Yes, that would add some overhead, but I don't see a way around it because we actually don't have control over the content of the OES texture, that is completely up to the device manufacturer. This is why it is a fast path, because it's use the decoded texture directly...

As for the new issue - I can't see it moving around on the mesh.. I presume the mesh is your ground plane. Are you only seeing it moving around in a VR headset? I ask because VR headsets do all sorts of prediction when moving the head which means the image could appear at the wrong location for a moment when moving the head quickly.

The only other artifacts I see are: 1) z-fighting caused by the geometry being too close to each other 2) culling making the ground disappear - I'm not sure why this is happening but this must be a Unity bug.

Anything related to static batching is nothing we have control over - that's all Unity side. All we do is assign the video texture to the material on the Mesh, which is simply a material change.

Does that help?

[tomeroberts91]

The mesh should be a screen like this:

but it ends up on the ground like that when deployed to an Android device

[tomeroberts91]

Also there is no VR headset, it's all on the Android device that it does this, works fine in Editor

[tomeroberts91]

Here is an example of what is happening in a video. It starts with Fast Path off and the screen is rendering normally in the correct position, however when I turn the fast path on, it renders flat on the ground (which in this case seems to be if the Plane object that makes the screen was at 0,0,0 pos 0,0,0 rot and 1,1,1 scale). This was done by activating and deactivating two media players when the buttons are clicked. https://1drv.ms/v/s!Au8PukIbIUbEgr9csRpoX36fTaXekg?e=IAiftB

My thought and looking through the code is that it could be something on the Native level not being marshalled across correctly. I don't know how the native render plugin works so I can only guess though

[Ste-RH]

A quick house-keeping update on this one.

We have managed to reproduce in house on a Nexus Shield tablet, but the issue does not show up on an Nexus 7 (2nd gen). As a possible short-term fix...changing the Graphics API to OpenGLES 2 seems to fix the issue.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had recent activity. If you wish to continue this issue, then please create a new issue as we do not monitor closed issues.

[Ste-RH]

I have tested v2 of AVPro Video with this issue and it still exists. I have also tried the very latest versions of Unity (2020 and 2021 alpha). The issue still happens.

I tracked it back to gl_ModelViewProjectionMatrix not carrying the transform set on 'PlaneFastPath' (in your sample project) in the vertex shader. I believe this is a bug beyond AVPro Video, and actually may lie in Unity as the issue does not happen when not using URP.

I am afraid we do not have a fix we can offer other than to switch to use OpenGLES2.

[tomeroberts91]

Sorry for a late reply, you know how work can overwhelm, especially at a time like this. But I was wondering if there was a way to (or if you already have) submit a bug to Unity about this problem? As OpenGLES2 doesn't work very well with URP in Linear colour space (which we need).