search

Unity-Technologies / com.unity.webrtc

WebRTC package for Unity
Other
753 stars 191 forks source link

[BUG]: Segmentation Fault Error on Linux Server Build with WebRTC-exp7 or above #791

Closed ovsky closed 1 year ago

ovsky commented 2 years ago

Package version

2.4.0-exp.7 or above

Environment

* OS: Ubuntu 20.04.4 LTS
* Unity version: 2021.3.5f1 LTS

Steps To Reproduce

  1. Build Unity project with WebRTC-exp7 or above for Linux Dedicated Server Platform.
  2. Try to run it on any Linux distribution.

Current Behavior

Loading application causes the Segmentation fault error when loading libwebrtc.so. 

[UnityMemory] Configuration Parameters - Can be set up in boot.config
    "memorysetup-bucket-allocator-granularity=16"
    "memorysetup-bucket-allocator-bucket-count=8"
    "memorysetup-bucket-allocator-block-size=4194304"
    "memorysetup-bucket-allocator-block-count=1"
    "memorysetup-main-allocator-block-size=16777216"
    "memorysetup-thread-allocator-block-size=16777216"
    "memorysetup-gfx-main-allocator-block-size=16777216"
    "memorysetup-gfx-thread-allocator-block-size=16777216"
    "memorysetup-cache-allocator-block-size=4194304"
    "memorysetup-typetree-allocator-block-size=2097152"
    "memorysetup-profiler-bucket-allocator-granularity=16"
    "memorysetup-profiler-bucket-allocator-bucket-count=8"
    "memorysetup-profiler-bucket-allocator-block-size=4194304"
    "memorysetup-profiler-bucket-allocator-block-count=1"
    "memorysetup-profiler-allocator-block-size=16777216"
    "memorysetup-profiler-editor-allocator-block-size=1048576"
    "memorysetup-temp-allocator-size-main=4194304"
    "memorysetup-job-temp-allocator-block-size=2097152"
    "memorysetup-job-temp-allocator-block-size-background=1048576"
    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"
    "memorysetup-temp-allocator-size-background-worker=32768"
    "memorysetup-temp-allocator-size-job-worker=262144"
    "memorysetup-temp-allocator-size-preload-manager=262144"
    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gfx=262144"
Preloaded 'lib_burst_generated.so'
Preloaded 'libwebrtc.so'
Segmentation fault

...and server crashes at this point.

Expected Behavior

The package should be loaded without any problems, like with removed WebRTC library:

    (...)
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gfx=262144"
Preloaded 'lib_burst_generated.so'
Initialize engine version: 2021.3.5f1 (40eb3a945986)

Anything else?

I have installed all dependencies and packages that may be needed to make WebRTC working properly. This bug occurs on any build settings (Mono/.NET Framework/.NET Core/Burst and no-Burst). Previoulsy there will be no problem to run

Packages above exp.6 - 7, 8, and 9 causes the same problem.

Interesting fact

After removing the libwebrtc.so package from build's Data>Plugins the server works properly (as shown above) - obviously, without WebRTC functionalities.

karasusan commented 2 years ago

Is it same issue? https://github.com/Unity-Technologies/com.unity.webrtc/issues/753

ovsky commented 2 years ago

No, it's another case, because in my issue the application just won't start - it is unable to load the libwebrtc.so library, so there aren't problems with signaling or streaming for now. But it's a bit similar - beacuse using the older version of the package also worked for the suggested issue reporter as well.

karasusan commented 2 years ago

Can I have more detail of the Segmentation fault? It might put the crash log.

ovsky commented 2 years ago

So, as this is a segmentation error, it may will be best if I leave you a valgrind logs (from Ubuntu WSL this time).

  1. from basic app verison: valgrind.log
  2. from app with debug mode and script debugging: valgrind-debug.log

The Unity application does not leave any crash logs on its own.

Edit: Ok, I think it won't be super helpful in this situation, but I will also leave the strace logs:

  1. strace.log
  2. strace-debug.log
karasusan commented 2 years ago

Thank you for sharing the detail. We will check it.

karasusan commented 2 years ago

memo: WRS-388

ZenBre4ker commented 1 year ago

Can confirm, that this only happens on the "Dedicated Server Build".

Libwebrtc.so just can't be loaded and leads to a Segmentation fault.

Still with the newest version 3.0.0.-pre.1 274791c

ZenBre4ker commented 1 year ago

I started the server with LD_DEBUG=all @karasusan Maybe this debug-log of the preload can help. Debug-Libwebrtc.so.txt

It seems to miss references in the libktx_unity.so and libwebrtc.so like:

These are only missing in the libktx_unity.so

karasusan commented 1 year ago

@ZenBre4ker Thank you for the detail. I have a question. The dedicated server build strips the graphic feature from the binary. This app cannot render the graphics. Is this OK for your use-case?

ZenBre4ker commented 1 year ago

@karasusan Yeah that's fine. The server build in my use-case only needs to receive and relay webrtc-streams. In general a headless build doesnt need graphics, I assumed.

karasusan commented 1 year ago

@ZenBre4ker All right. I added the task into our backlog.

keepingitneil commented 1 year ago

Also seeing this for a headless linux build. Adding this comment to help inform priority. Thanks!

davidyrgilbert commented 1 year ago

This is still quite a breaking issue for us, is there any new information regarding the segfault?

karasusan commented 1 year ago

I'm surprised that there are many request for this issue. I'm curious to see the main use case using the headless linux build.

gtk2k commented 1 year ago

Using cloud rendering.

davidyrgilbert commented 1 year ago
karasusan commented 1 year ago

@gtk2k I haven't checked the video streaming feature for the dedicated server, does it work?

gtk2k commented 1 year ago

@karasusan It's not headless at the moment, but we're planning to move to headless in the future if it works.

karasusan commented 1 year ago

@gtk2k I commented about the dedicated server, I think it might not work as you expected. https://github.com/Unity-Technologies/com.unity.webrtc/issues/791#issuecomment-1329999315

ZenBre4ker commented 1 year ago

@karasusan Small Update concerning the Segmentation Fault. We made a small test-project only consisting of the webrtc sample and built via Unity UI directly a dedicated Server as linux .x86_64 and as windows .exe. While the dedicated Server for Windows works, stating this:

[UnityMemory] Configuration Parameters - Can be set up in boot.config
    "memorysetup-bucket-allocator-granularity=16"
    "memorysetup-bucket-allocator-bucket-count=8"
    "memorysetup-bucket-allocator-block-size=4194304"
    "memorysetup-bucket-allocator-block-count=1"
    "memorysetup-main-allocator-block-size=16777216"
    "memorysetup-thread-allocator-block-size=16777216"
    "memorysetup-gfx-main-allocator-block-size=16777216"
    "memorysetup-gfx-thread-allocator-block-size=16777216"
    "memorysetup-cache-allocator-block-size=4194304"
    "memorysetup-typetree-allocator-block-size=2097152"
    "memorysetup-profiler-bucket-allocator-granularity=16"
    "memorysetup-profiler-bucket-allocator-bucket-count=8"
    "memorysetup-profiler-bucket-allocator-block-size=4194304"
    "memorysetup-profiler-bucket-allocator-block-count=1"
    "memorysetup-profiler-allocator-block-size=16777216"
    "memorysetup-profiler-editor-allocator-block-size=1048576"
    "memorysetup-temp-allocator-size-main=4194304"
    "memorysetup-job-temp-allocator-block-size=2097152"
    "memorysetup-job-temp-allocator-block-size-background=1048576"
    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"
    "memorysetup-temp-allocator-size-background-worker=32768"
    "memorysetup-temp-allocator-size-job-worker=262144"
    "memorysetup-temp-allocator-size-preload-manager=262144"
    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gfx=262144"
Initialize engine version: 2021.3.19f1 (c9714fde33b6)
[Subsystems] Discovering subsystems at path D:/Test_Projekt/WebRtc_Sample_Test/Builds/Windows/WebRtc_Sample_Test_Data/UnitySubsystems
Forcing GfxDevice: Null
GfxDevice: creating device client; threaded=0; jobified=0
NullGfxDevice:
    Version:  NULL 1.0 [1.0]
    Renderer: Null Device
    Vendor:   Unity Technologies
ERROR: Shader Sprites/Default shader is not supported on this GPU (none of subshaders/fallbacks are suitable)
Microsoft Media Foundation video decoding to texture disabled: graphics device is Null, only Direct3D 11 and Direct3D 12 (only on desktop) are supported for hardware-accelerated video decoding.
ERROR: Shader Sprites/Mask shader is not supported on this GPU (none of subshaders/fallbacks are suitable)
ERROR: Shader GUI/Text Shader shader is not supported on this GPU (none of subshaders/fallbacks are suitable)
UnloadTime: 0.546800 ms
ERROR: Shader UI/Default shader is not supported on this GPU (none of subshaders/fallbacks are suitable)

The Linux dedicated Server build (no matter if done with Windows or Linux) grafik throws this:

[UnityMemory] Configuration Parameters - Can be set up in boot.config
    "memorysetup-bucket-allocator-granularity=16"
    "memorysetup-bucket-allocator-bucket-count=8"
    "memorysetup-bucket-allocator-block-size=4194304"
    "memorysetup-bucket-allocator-block-count=1"
    "memorysetup-main-allocator-block-size=16777216"
    "memorysetup-thread-allocator-block-size=16777216"
    "memorysetup-gfx-main-allocator-block-size=16777216"
    "memorysetup-gfx-thread-allocator-block-size=16777216"
    "memorysetup-cache-allocator-block-size=4194304"
    "memorysetup-typetree-allocator-block-size=2097152"
    "memorysetup-profiler-bucket-allocator-granularity=16"
    "memorysetup-profiler-bucket-allocator-bucket-count=8"
    "memorysetup-profiler-bucket-allocator-block-size=4194304"
    "memorysetup-profiler-bucket-allocator-block-count=1"
    "memorysetup-profiler-allocator-block-size=16777216"
    "memorysetup-profiler-editor-allocator-block-size=1048576"
    "memorysetup-temp-allocator-size-main=4194304"
    "memorysetup-job-temp-allocator-block-size=2097152"
    "memorysetup-job-temp-allocator-block-size-background=1048576"
    "memorysetup-job-temp-allocator-reduction-small-platforms=262144"
    "memorysetup-temp-allocator-size-background-worker=32768"
    "memorysetup-temp-allocator-size-job-worker=262144"
    "memorysetup-temp-allocator-size-preload-manager=262144"
    "memorysetup-temp-allocator-size-nav-mesh-worker=65536"
    "memorysetup-temp-allocator-size-audio-worker=65536"
    "memorysetup-temp-allocator-size-cloud-worker=32768"
    "memorysetup-temp-allocator-size-gfx=262144"
Preloaded 'libwebrtc.so'
Unable to load player prefs
Segmentation fault (core dumped)

So maybe theres something wrong with linux dedicated server support directly or the compilation of libwebrtcs.so compared to the webrtc.dll on windows?

karasusan commented 1 year ago

@ZenBre4ker Thank you for checking. Maybe libwebrtc.so has something wrong.

LazyWood commented 1 year ago

Excuse me, when will the headless server be supported?

karasusan commented 1 year ago

@LazyWood Sorry but we can't say the date when it will be supported.

ZenBre4ker commented 1 year ago

@karasusan Further investigation shows, that commenting out this line

if (!vulkan->AddInterceptInitialization(InterceptVulkanInitialization, nullptr, 0))
    {
        RTC_LOG(LS_INFO) << "AddInterceptInitialization failed.";
    }

https://github.com/Unity-Technologies/com.unity.webrtc/blob/main/Plugin%7E/WebRTCPlugin/UnityRenderEvent.cpp#L193 lets the program properly initialize before crashing later. Valgrind states, that there is an invalid read of size 8.

The crash later is probably due to the missing vulkanInterception, because the code is hidden inside the UnityPlayer.so, which i dont know how to get Debug Symbols for.

So maybe its rather a problem with vulkan on a dedicated server?

karasusan commented 1 year ago

@ZenBre4ker There is a possibility. Graphics features are stripped when building a dedicated server, so might occurrs a crash on the line which you mentioned.

cakeslice commented 1 year ago

@karasusan Still getting this issue on a dedicated server linux build even with the merged changes from #887:

symbol lookup error: undefined symbol: UnitySetGraphicsDevice (fatal) symbol lookup error: undefined symbol: UnityRenderEvent (fatal) symbol lookup error: undefined symbol: UnityGetAudioEffectDefinitions (fatal) symbol lookup error: undefined symbol: UnityRenderingExtEvent (fatal) symbol lookup error: undefined symbol: UnityRenderingExtQuery (fatal) symbol lookup error: undefined symbol: UnityShaderCompilerExtEvent (fatal)

For my use case, I only want to use the data channel so I don't need audio, video or rendering of any kind.

ZenBre4ker commented 1 year ago

@cakeslice The changes from #887 only change source files, they still need to be compiled into a shared library to work. Also your errors are normal and dont mean that it doesnt work. I fixed only the segmentation fault due to missing vulkan, not any missing symbols.

cakeslice commented 1 year ago

@ZenBre4ker You're right, after compiling it works now. Thank you!