Loading high bit-rate MP4 causes dropped Unity frame

[MattCoalman]

I am working on an application to be used in movie theater, and i need to be able to switch video file immediatly between two possible videos.

Here is my setup :

1. I have 3 AVPro Media player on my scene (Player A, B and C)
2. I load a media from the A media player
3. I launch Play() on the A media player
4. I load a media from on B
5. When the ReadyToPlay event is raised from B, i load a media on C

My problem: A little image freeze (or FPS decrease) is noticable on player A when loading the media on B and C. I don't understand why. I think the media playing should have a higher priority than media opening.

I am currently trying to decide if I need to buy the AVPro Video plugin to do my task, so, is this behaviour normal? Am I doing things properly? How to load media during another player playing without FPS loss?

Some info : PC with I7 processor Windows 10 NVMe drive GTX1050 Unity 2020 AVPro Media Player Trial

Thanks a lot.

[kahnivore]

Hi there,

Have you had a look at our Demo_Playlist scene? It uses two Media Players to load and switch between content instead of three, which could help your application be more performant. But this will also depend on your hardware and media.

[MattCoalman]

Hi there,

Have you had a look at our Demo_Playlist scene? It uses two Media Players to load and switch between content instead of three, which could help your application be more performant. But this will also depend on your hardware and media.

Hello. Thank for the answer.

I've checked the Demo_Playlist scene, but the two medias url are put directly in the editor scene and not by script, and the media are loaded before any playing occurs, so I don't see how this demo can help me. Am I missing something?

Thanks.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[MattCoalman]

Hello. Please don't close it. I still need to load a media during playing another without having fps loss.

[kahnivore]

Hi there, sorry for the slow response. I suggested the playlist demo to show that two Media Players could be used over three, which might help with performance in your scene. That is, load/play media in A, then load in B while A is playing, and then when B plays, load the next one in A. Do you still get the lag if you use two Media Players over three?

[MattCoalman]

Thanks for you answer. I use 3 players because I need to be able to choose at the last moment which video to play next, and need to start immediatly (kind of a seemless interactive movie). And because it will be screened at a public venue with high quality video files, I am currently trying to load high bitrate movie files, and i am experiencing lag when I use 100MBps H264/5 files. Which I don't quite understand because the buffering takes only 500ms, but somehow, the MediaPlayer put a higher priority on buffering a file on the the player B, and not on keeping the current movie playing correctly on player A. I would prefer the buffering take longer to make the currently playing movie stay smooth.

So yes, it looks to usually works fine with 50MBps files buffering, but this doesn't look very reliable to me, and I feel I will be afraid to see frame drops each time I will load another movie.

Isn't there a way, or a parameter to tell the mediaplayer "Hey, buffer this new file, but chill, and take the time you need to keep the currently movie playing perfectly, because that's your top priority."?

Thanks a lot.

[kahnivore]

What resolution, bitrate and framerate are your videos? Could you have a look at your task manager performance tab while running your application to see if the GPU is maxing out? Suspect this might be the case. If so, you could try re-encoding your media with a fast decode profile on FFmpeg which might help.

[MattCoalman]

Hello there,

Video A (the one playing): Small 6sec video H264 12MB, 1080p, 15Mbps, 30fps loop video (i-frame only) Video B (the one buffering) : 30sec video H265, 360MB, 1080p, 100Mbps, 29.97fps

When playing Video A :

• CPU : <2%
• GPU : 3D :<40%, Decode : <15%
• SSD : <2%

When playing video A & buffering video B: (duration of buffering: 300ms) 🡆 This is when frame drop on Video A are visible

• SSD : Small peak to 5%
• Nothing else really change, or too quick to be noticed within 300ms

Edit: Aditionnal info : When both Video A & B are playing: 🡆 No frame drop once video B is buffered

• SSD: Some peaks at 15%
• CPU: <10%
• GPU Decode : ≈45-50%

[MattCoalman]

I tested with another computer with a GTX 1080 (this computer is super stable, and dedicated to be used on live performances), and i have the same problem. (GPU Decode is arround 30% when both Videos A and B are playing)

I made made another test : I was playing video A on VLC, and launched video B on another instance of VLC : No frame drop. I even tested with the 400Mbps version of Video B (30s, 1,4GB), still no frame drop.

So it looks like this is not a computer related performance issue.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[MattCoalman]

Hi. I'm still looking for a solution, please don't close.

[Chris-RH]

Could you perhaps send us a sample reproducible project to unitysupport@renderheads.com? Please include the issue number (1111) in the subject

[Chris-RH]

Hi @MattCoalman, If this is still a problem, then it would be really helpful if you could send us a sample reproducible project to unitysupport@renderheads.com? Please include the issue number (1111) in the subject

[MattCoalman]

Hi @MattCoalman, If this is still a problem, then it would be really helpful if you could send us a sample reproducible project to unitysupport@renderheads.com? Please include the issue number (1111) in the subject

Done! Thanks a lot

[Chris-RH]

Thank you, we'll look at that today

[Chris-RH]

Hi @MattCoalman, Thank you for your email. Could you please:

1. Try ticking the 'low latency' option, and also try unticking the 'text track support' option as that's not needed

2. Use a profile so that we can see how it works for you Here is ours:

3. Try making an IL2CPP build (better perf and garbage collection)

4. In the Events section in the MediaPlayer component, try settings the "triggered events" to "nothing" to save some perf

5. Try ticking the "Disable Logging" option in Global section (logging is slow)

6. If audio is not required, set "audio output" mode to "None" instead of "Unity"

[AndrewRH]

@MattCoalman in addition to the above, we've also found a few performance issues internally and are currently investigating - thanks to your demo scene. We'll keep you updated, but hopefully the above steps do add some improvement. Also it would be great to get a screenshot of your profile results (as above) so we can compare if yours is very different from what we're seeing.

Thanks,

[MattCoalman]

Hi @Chris-RH and @AndrewRH ! Here is my capture. (I can see CPU usage going really up, but i can't really see the same percentage than you in the overview)

I will try tomorrow the other option settings you are asking me to do.

[MattCoalman]

Hi @MattCoalman, Thank you for your email. Could you please:

1. Try ticking the 'low latency' option, and also try unticking the 'text track support' option as that's not needed
2. Use a profile so that we can see how it works for you
3. Try making an IL2CPP build (better perf and garbage collection)
4. In the Events section in the MediaPlayer component, try settings the "triggered events" to "nothing" to save some perf
5. Try ticking the "Disable Logging" option in Global section (logging is slow)
6. If audio is not required, set "audio output" mode to "None" instead of "Unity"

Hi ! For your information, none of the suggestions above helps to avoid frame drop. And you're right, the whole application has frame drop, not only the first video. I tried to make an object constantly moving, and the animation of this object is stopped some frames while video is loading.

[Chris-RH]

Hi @MattCoalman , thank you so much, this has been very helpful. We'll be investigating further.

[AndrewRH]

Hi @MattCoalman

Here is a new DLL for you to test. Please copy this into your AVProVideo/Runtime/Plugins/Windows/x86_64 folder (with Unity closed)

AVProVideo.zip

See if you get better performance with this? Thanks

[MattCoalman]

Hi @MattCoalman

Here is a new DLL for you to test. Please copy this into your AVProVideo/Runtime/Plugins/Windows/x86_64 folder (with Unity closed)

AVProVideo.zip

See if you get better performance with this? Thanks

@AndrewRH I have this error :

[AVProVideo] Plugin version number 2.5.0f2-trial doesn't match the expected version number 2.3.2.  It looks like the plugin didn't upgrade correctly.  To resolve this please restart Unity and try to upgrade the package again.
UnityEngine.Debug:LogError(Object)

Edit: Ok, i was able to use it by modifying the ExpectedPluginVersion in Helper.cs

[MattCoalman]

Hi @AndrewRH I tried and the new DLL doesn't seems to helps with the performance issue. Feel free to give me other versions to try. Thanks

[Chris-RH]

OK, thank you for trying it. We'll have another look into this

[AndrewRH]

@MattCoalman oh that's a shame. There should be SOME performance improvement - we're certainly seeing it here. Could you show us how your profiler looks now? We're especially interested in the call-stack for the spikes related to AVPro Video.

Have you tried in a standalone build or is this all in the editor?

Thanks,

[MattCoalman]

I can't really see a big change. But hear is my profiler result:

Some result from the peak of the CPU:

And maybe some interresting result from the GPU:

And if you add a simple Sphere in the scene, and add this code:

    public GameObject sphere = null;

    private float sphere_pos_x = 0;
    void MoveSphere()
    {
        if (sphere_pos_x > 8.25)
            sphere_pos_x = -8.25f;
        else
            sphere_pos_x += 0.1f;

        sphere.transform.position = new Vector3(sphere_pos_x, 0);
    }

    void Update()
    {
        MoveSphere();
}

You will be able to see very easily any fps drop.

[AndrewRH]

Still looking into this

[Chris-RH]

Hi @MattCoalman, We are still looking into this. In the mean time could you try updating to version 2.5.2 please. Hopefully you should see a bit of improvement in performance. Let us know if it looks any better. Kind regards, Chris

[MattCoalman]

Hi @Chris-RH , I'm glad to hear that you are still on it :) This problem is really blocking me, and is decisive for the viability of my project. (I've tried the last version 2.5.2, and there is still a large drop frame, and improvement is not really noticable)

[AndrewRH]

Thanks, we're continuing to look into it

[MattCoalman]

@AndrewRH Thanks a lot! Also, i'm not sure this is useful, but here is a video capture of the frame drop: https://www.youtube.com/watch?v=NGMEaYkS0EI

[MattCoalman]

Hello. Do you know any way to bypass this problem? Maybe preload stream in memory before load video player? Another question, why the ticket is still in triage? Isn't a bug? (because you too notices performance issue) Thanks.

[AndrewRH]

Yes we're still looking into this. We're a small team so these things sometimes take time.

Thanks for your patience.

[MattCoalman]

Hello. Any news? I still cannot open large video without a significant framerate drop. Thanks

[Chris-RH]

Hi @MattCoalman,

I'm very sorry. Unfortunately the dev who was looking into this has left. From what I gather, it is quite a big job as it involves a change of workflow. It is still on the job list though. We are a very small team, so I'm not sure exactly when its going to get done.

Kind regards, Chris