search

Unity-Technologies / com.unity.webrtc

WebRTC package for Unity
Other
753 stars 191 forks source link

[REQUEST]: Video codec for Meta Quest 2 #838

Open karasusan opened 1 year ago

karasusan commented 1 year ago

Detail of the issue bellow. https://github.com/Unity-Technologies/UnityRenderStreaming/issues/716

shofixti244 commented 1 year ago

Just commenting to say this is an important issue for me.

dhruvaura commented 1 year ago

Yes this is a most important issue for many developer. Eagerly waiting for this get fix.

half-oval commented 1 year ago

Waiting for this solution. This can solve performance issue of Meta Quest when we building some streaming solutions. Request to implement codec asap.

karasusan commented 1 year ago

memo: WRS-378

karasusan commented 1 year ago

Related issue #876.

kar-re commented 11 months ago

Hi @karasusan! I'm quite interested in this fix as well, and I am interested in contributing to fix this. Do you have any pointers regarding where to look or what to implement in order to mitigate this issue?

BR, Kar-re

karasusan commented 11 months ago

@kar-re The problem is that the codecs implemented by MediaCodec cannot be referenced on Meta Quest 2. Android codec factory class is implemented by AndroidCodecFactoryHelper.

This class calls the MediaCodec API via a Java class. First, it is necessary to investigate whether the information expected by this API can be obtained.

kar-re commented 11 months ago

I see, thank you for the info @karasusan! I'll give it a go and submit a PR if I manage to get it working.

kar-re commented 11 months ago

Hi everyone! Just wanted to share some findings. Unfortunately I haven't been able to solve the issue in its entirety, but I managed to build a version of the application I'm developing without the stuttering I experienced previously. I thought about writing a custom VideoEncoder for webrtc based on this repo, but too much work for initial experimentation.

The steps taken by me were:

  1. Build the native android plugin on my own
  2. Change from Vulkan to OpenGLES3 (It crashed with vulkan)

I've tried to experiment with different codecs and it seems to affect the streamed performance. For my use case it's not really dependent on the recieved fps, but it might be worth knowing. Here is my built .aar if anyone wants to experiment.

This also makes me think it has to do with vulkan compatability somehow.