Significant drop in FPS with Playback on Vulkan

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback

https://www.renderheads.com/products/avpro-video/

224 stars 27 forks source link

Significant drop in FPS with Playback on Vulkan #1871

Closed cieraej closed 1 week ago

cieraej commented 1 month ago

Issue

There is a significant loss in frame rate (around 15 frames) when playing a video on the Quest 2 using Vulkan. This issue was not present when using OpenGLES with the same setup.

Setup (please complete the following information):

Unity version: 2022.3.4 (upgrading to 2022.3.11) AVPro Video version (number and edition (trial/core/ultra/enterprise)): 3.0.0, Ultra Operating system version: Android Device model: Quest 2 Video specs (resolution, frame-rate, codec, file size): 2500x1248 ll360 H264.mp4

Other Notes

The issue occurs only with Vulkan. No significant frame rate issues were experienced with OpenGLES using the same version of Unity and AVPro.

Before/After Video Playback OVR Metrics Tool (Vulkan) (Drops 15 Frames)

Before/After Video Playback OVR Metrics Tool (OPENGLES + OES) (Drops 3 frames)

Chris-RH commented 1 month ago

Is this repeatable in a new project running Unity video player? Is this repeatable in a new project running only AVPro video mediaplayer demo (no other assets, such as XR)?

cieraej commented 1 month ago

I can make a test project that is fairly empty, however, i would need an xr package to ensure I can have it run on our target device (Oculus Quest)

cieraej commented 3 weeks ago

So I ran a build and it looked good when I made a test project. The issue seems to be that we have a custom shader that assumes OES for playback. Is there any initiative to get OES playback on Android/Vulkan? That looks like it would solve our FPS drop.

Ste-RH commented 3 weeks ago

OES on Vulkan is a desire to support, but currently it is not being worked on and there is no time-frame for it's potential delivery to AVPro Video

Ste-RH commented 3 weeks ago

Do you see the same issue using one of the supplied sample videos? (BigBuckBunny or the RenderHeads videos)

Ste-RH commented 2 weeks ago

I think this has been addressed in v3.0.2. Can you update and verify please @cieraej ?

Chris-RH commented 2 weeks ago

Any luck @cieraej ?

cieraej commented 2 weeks ago

I have had really mixed results with stability after the new update! Notably, I had one build that was pretty stable, but I have been unable to replicate that stability since then. The primary issue appears to be related to seeking and toggling between streams and local files. So I have a few questions to investigate this:

Do you have any recommendations for optimal AVPro settings when using Vulkan?
Which player (ExoPlayer or MediaPlayer) would you recommend?

For additional context our local files are 2500x1248 resolution and formats tend to be ll360 H264.mp4, and our stream formats are m3u8 and mpds.

Thanks again :)

Ste-RH commented 2 weeks ago

We have made some fixes in the direction of Vulkan, might be worth grabbing the latest version @cieraej to see if that helps (v3.0.3).

No special settings to note
We have seen faster seeking using the MediaPlayer API path, but streaming support is not as solid. I would always choose ExoPlayer over MediaPlayer unless there is a compelling reason not to based on a use-case requirements

If the original issue is addressed, please do let us know so we might close this issue - and open a fresh issue for anything else that is causing issue.