search

RenderHeads / UnityPlugin-AVProVideo

AVPro Video is a multi-platform Unity plugin for advanced video playback
https://www.renderheads.com/products/avpro-video/
235 stars 28 forks source link

Low display rate on macOS when using the Metal rendering API #516

Closed nhasson76 closed 3 years ago

nhasson76 commented 3 years ago

Describe the bug A clear and concise description of what the bug is.

When playing mp4 video on macOS from HTTPS or from local file and the frame rate drops a bit bellow ~40 FPS the video display rate decreases dramatically to 1-2 and the video playback looks almost frozen.

Device (which devices are you having the issue with - model, OS version number):

Mac Mini, Macbook Pro, Catalina 10.15.7

Media (tell us about your videos - number of videos, resolution, codec, frame-rate, example URLs):

1 video, 640x480, 24 FPS, mp4 https://spivi-cdn.s3.amazonaws.com/vod/1632/1444579/audio_video_640x480_2020-11-06_09-30-46_AHF3U.mp4

System Information:

AVPro Video: plugin v1.11.4 scripts v1.11.5 Target Platform: Standalone Unity: v2019.4.13f1 OSXEditor OS: Desktop - Macmini7,1 - Mac OS X 10.15.7 - English CPU: Intel(R) Core(TM) i5-4308U CPU @ 2.80GHz - 4 threads - 8192KB GPU: Intel Iris Graphics - Apple - Metal - 1536KB - 16384

Your Setup (please complete the following information):

To Reproduce

  1. Create a blank screen
  2. Add the FullscreenVideo prefab.
  3. Paste a valid URL into the Source Path property.
  4. Add some 3D objects such as terrains.
  5. Press play.
  6. Let the FPS go below 40 by activating the 3D objects.
  7. Check the display rate of the video.

Logs If applicable, add error logs to help explain your problem.

Screenshots If applicable, add screenshots to help explain your problem.

Videos If applicable, add a copy of your video or the URL

https://www.dropbox.com/s/dx867c9yqxyxh0g/AVPro-macOS-stall.MOV?dl=0

Please DO NOT LINK / ATTACH YOUR PROJECT FILES HERE

Instead email the link to us unitysupport@renderheads.com

nhasson76 commented 3 years ago

Another clue that might help you trace the source of this issue: In a previous version of AVPro I could play WebM files without any issue. It seems like the mp4 encoding is somehow related to the rendering thread of unity and GPU usage.

MorrisRH commented 3 years ago

Are you able to share your project? (if so please send it to unitysupport@renderheads.com and mention this issue).

nhasson76 commented 3 years ago

Yes, just sent you a custom package with instructions.

MorrisRH commented 3 years ago

Thanks! I can now reproduce the issue so will investigate. It only seems to affect the Metal renderer so if you can switch to OpenGLCore in the interim if it's holding you up.

nhasson76 commented 3 years ago

Hi Morris,

I just tried it out using OpenGL Core and nothing has been changed. I restarted the editor and even the PC itself and I get the same result.

Regards,

Nir Hasson,

SMART TRAINING SYSTEMS FOR STUDIOS AND FITNESS CLUBS

On Tue, Nov 10, 2020 at 4:27 PM Morris Butler notifications@github.com wrote:

Thanks! I can now reproduce the issue so will investigate. It only seems to affect the Metal renderer so if you can switch to OpenGLCore in the interim if it's holding you up.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/516#issuecomment-724736177, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNUHGUUORDN5YI25JXKHOTSPFEWHANCNFSM4TOMGTIQ .

MorrisRH commented 3 years ago

Seems IMGUI is causing the issue here. Profiling showed a large stall with Gfx.WaitForPresentOnGfxThread, effectively holding everything up. As there is nothing in the plugin that would cause such a stall I believe the issue is with Unity.

I ran some tests using our DisplayUGUI and ApplyToMaterial components and saw no issues so I recommend you use something other than IMGUI.

nhasson76 commented 3 years ago

Well, in my production project we don’t use any of these. Instead we need to have the finalized texture as a unity texture so we render it using a shader to a render target and use the render target for display.

On Tue, 10 Nov 2020 at 18:08 Morris Butler notifications@github.com wrote:

Seems IMGUI is causing the issue here. Profiling showed a large stall with Gfx.WaitForPresentOnGfxThread, effectively holding everything up. As there is nothing in the plugin that would cause such a stall I believe the issue is with Unity.

I ran some tests using our DisplayUGUI and ApplyToMaterial components and saw no issues so I recommend you use something other than IMGUI.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/RenderHeads/UnityPlugin-AVProVideo/issues/516#issuecomment-724801538, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABNUHGTBZ74H4WQXSA6EHKLSPFQRHANCNFSM4TOMGTIQ .

-- Regards,

Nir Hasson,

SMART TRAINING SYSTEMS FOR STUDIOS AND FITNESS CLUBS

nhasson76 commented 3 years ago

I tested the same project without the IMGUI component and got the same results. I added a custom script to show the display rate to avoid performance biasing caused by the inspector. Tested also with Unity 2019.4.14f1 and the issue persists.

MorrisRH commented 3 years ago

It looks like the hardware isn't up to the challenge of rendering that many triangles at once.

I've now tested on an old MacBook Pro (late 2013) with similar hardware, most importantly the Intel Iris GPU. I'm seeing the same huge stall with Gfx.WaitForPresentOnGfxThread with and without the video running. The video update is triggered from the main thread so when it gets blocked like that, frames get lost which leads to the low display rate your seeing.

nhasson76 commented 3 years ago

I guess you're right. What I don't get is how it worked smoothly when it played a WebM file before. Was it a software decoder? I also tested with the native unity video player on macOS and got the same results and the console showed an error saying "Player having difficulty streaming video https://xxxxxxxx.mp4"

MorrisRH commented 3 years ago

I'm not sure how you had a WebM file playing with AVPro Video, we've never supported it on macOS. However there is no hardware support for VP8 so it was likely using software decode.

nhasson76 commented 3 years ago

I probably got confused, I guess I tested it with the native unity video player. Anyways, seems like this is a low-level hardware issue. I posted in the unity forums, I'll see if I'll get any useful information from their end.

nhasson76 commented 3 years ago

I chose the WebM path for now until a fix will be available. I replaced the AVPro with the native unity video player and got very good results in terms of playback display rate. Even when the FPS goes below 18 the video playback remains steady. The main downside for this path is the Untiy doesn't really stream the WebM, it downloads the whole file and then plays it...

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] commented 3 years ago

This issue has been automatically closed because it has not had recent activity. If you wish to continue this issue, then please create a new issue as we do not monitor closed issues.