Unity-Technologies / com.unity.webrtc

WebRTC package for Unity
Other
739 stars 185 forks source link

fix: Change UnityVulkanPluginEventConfig parameters for vulkan API #889

Closed karasusan closed 1 year ago

karasusan commented 1 year ago

This PR fixes the performance issue of video streaming when using vulkan graphics API on Linux with Unity 2022.2.

This issue is able to replicated by this condition below.

The cause of this issue is handling errors occurred by vulkan synchronization between GPU and CPU. In addition, The timing of accessing rendering queue is another reason. We fixed the timing with callback of UnityVulkanInterface::AccessQueue.

karasusan commented 1 year ago

When I run Unity Render Streaming with this change, it freezes on Ubuntu 20.04.

BrianHarrisonUnity commented 1 year ago

I believe we will need to include a fix from. #888 PR that resolves an infinite hang when calling ResetSync.

I made a branch with this change called vulkan-commandbuffer-experimental-fix that is working for me now on Ubuntu 20.04 when testing with the package samples.

karasusan commented 1 year ago

@BrianHarrisonUnity I merged your fix into this branch. Thank you.

BrianHarrisonUnity commented 1 year ago

Greetings @karasusan!

After testing around with this branch, I found a crashing issue occuring infrequently. I tracked down the issue, and it was happening due to calling the AccessTexture API during the OnRenderEvent with the event configured for exclusive queue access. The AccessTexture API cannot happen during events with exclusive queue access.

To resolve this issue, I created a new branch called, experimental/vulkan-queue-access-submit. I think this branch resolves the remaining issues. Also, since it no longer requires flushing the command buffer prior to the OnRenderEvent, I think the performance should be better as well!

karasusan commented 1 year ago

@BrianHarrisonUnity I checked the new branch experimental/vulkan-queue-access-submit.

PeerConnection sample in WebRTC package works well with this fix. However, when checking Broadcast sample in Unity Render Streaming package, I found video streaming stops for a while. We can see the issue in this video from 30 sec. Please pay attention to the mouse position in the video.

https://user-images.githubusercontent.com/1132081/229075158-cc7c531c-f4f4-42dc-839a-fe7f18a2d6a4.mp4

BrianHarrisonUnity commented 1 year ago

@karasusan I think I have seen the stream hiccup before, even on an old version of Unity & the package. In my case, it seemed to only happen if my machine was resource starved, and the client was unable to keep up temporarily. From the video, it looks like the sender is still running in the background, and it was only on the client side that the stream hiccuped.

Do you consistently see that issue, and if you try in release, or on a standalone build, does it still happen? Also, if you run it again, could you check what the WebRTC stats are showing when that happens?