search

adamrehn / ue4-docker

Windows and Linux containers for Unreal Engine 4
https://docs.adamrehn.com/ue4-docker/
MIT License
789 stars 173 forks source link

Failed to get CEF3 DLL handle #57

Open galeone opened 5 years ago

galeone commented 5 years ago

Output of the ue4-docker info command:

ue4-docker version:         0.0.38 (latest available version is 0.0.38)
Operating system:           Windows Server 2019 Datacenter Evaluation Version 1809 (OS Build 17763.379)
Docker daemon version:      19.03.3
NVIDIA Docker supported:    No
Maximum image size:         200GB
Available disk space:       374.14 GiB
Total system memory:        19.53 GiB physical, 35.29 GiB virtual
Number of processors:       10 physical, 20 logical

I'm using the docker images I build from a CI pipeline.

I have 2 different jobs, both of them with the same error.

The first job just executes:

ue4 test --all

The second instead:

ue4 package

Both of them fail because of the same issue:

[2019.11.04-16.28.17:961][  0]LogCEF3Utils: Error: Failed to get CEF3 DLL handle for ../../../Engine/Binaries/ThirdParty/CEF3/Win64/libcef.dll: The operation completed successfully. (0)
[....]

LogWindows: Error: begin: stack for UAT
  LogWindows: Error: === Critical error: ===
  LogWindows: Error:
  LogWindows: Error: Fatal error!
  LogWindows: Error:
  LogWindows: Error: Unhandled Exception: 0xc06d007e
  LogWindows: Error:
  LogWindows: Error: [Callstack] 0x00007ffd8e6b9129 KERNELBASE.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd60965456 UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd6092456c UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd608fcabc UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd608e22d2 UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd608d787b UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd60901d6f UE4Editor-WebBrowser.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd60889d76 UE4Editor-WebBrowserWidget.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd7d86b4ef UE4Editor-Core.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd83118fa7 UE4Editor-Projects.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd83129403 UE4Editor-Projects.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd83118d6e UE4Editor-Projects.dll!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a59d9c3 UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a5a49f1 UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a59a667 UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a59a9fa UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a5ab92c UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ff62a5ad736 UE4Editor-Cmd.exe!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffd91587974 KERNEL32.DLL!UnknownFunction []
  LogWindows: Error: [Callstack] 0x00007ffdc093a251 ntdll.dll!UnknownFunction []
  LogWindows: Error:
  LogWindows: Error: end: stack for UAT
Took 21.1138611s to run UE4Editor-Cmd.exe, ExitCode=3
ERROR: Cook failed.
       (see C:\Users\ContainerAdministrator\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+UnrealEngine\Log.txt for full exception trace)
AutomationTool exiting with ExitCode=25 (Error_UnknownCookFailure)
BUILD FAILED

Do you have any guess?

Running tests and packing locally (with the same engine) works fine, this happens only when using containers.

Thanks!

adamrehn commented 5 years ago

Hmmm, it's possible that CEF requires some additional DLL files that aren't present in the container's filesystem. I'll attempt to reproduce this error using a blank project with a WebBrowserWidget UI element and investigate further.

galeone commented 5 years ago

Thanks! Additional info: because of the bug of Docker for windows, I had to exclude from the engine compilation the debug symbols and templates (I don't know if it helps)

galeone commented 5 years ago

I have a guess: the build tool is looking at the wrong path.

I'm inside the container and there is the file libcef.dll inside the path c:\UnrealEngine\Engine\Binaries\ThirdParty\CEF3\Win64.

Looking at the error, it looks like to tool is looking for the DLL in the relative path: ../../../Engine/Binaries/ThirdParty/CEF3/Win64/

And unless the Gitlab runner placed the repository inside a subfolder of c:\UnrealEngine (and it doesn't, from what I see the repo is placed into c\builds\<projectname>\<reponame>), going back of 3 levels doesn't enter the c:\UnrealEngine folder, so ../../../Engine should be something like ../../../UnrealEngine/Engine IMHO (but I can be wrong)

However, since the error is about a handler, maybe the DLL has been correctly found (as you said) but there are other DLLs needed by the libcef.dll not found in the system (I was trying to use dependency walker inside the container, but I'm not proficient enough with Windows or Powershelll)

galeone commented 4 years ago

I can't help with Windows (altough is the one I need the most right now). But I checked with a Linux container and, yes, CEF also in Linux has some missing library.

Here's the output of ldd ./Engine/Binaries/ThirdParty/CEF3/Linux/libcef.so

        linux-vdso.so.1 (0x00007ffe36393000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe3372f4000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe3370f0000)
        libgobject-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fe336e9c000)
        libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fe336b85000)
        libfontconfig.so.1 => /usr/lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007fe336940000)
        libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007fe33668c000)
        libpangocairo-1.0.so.0 => not found
        libcairo.so.2 => not found
        libpango-1.0.so.0 => not found
        libnss3.so => not found
        libnssutil3.so => not found
        libsmime3.so => not found
        libnspr4.so => not found
        libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007fe336354000)
        libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007fe336144000)
        libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007fe335f3a000)
        libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007fe335d28000)
        libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007fe335b22000)
        libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fe335918000)
        libXss.so.1 => /usr/lib/x86_64-linux-gnu/libXss.so.1 (0x00007fe335714000)
        libXcomposite.so.1 => /usr/lib/x86_64-linux-gnu/libXcomposite.so.1 (0x00007fe335511000)
        libasound.so.2 => not found
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe335173000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe334f56000)
        libXdamage.so.1 => /usr/lib/x86_64-linux-gnu/libXdamage.so.1 (0x00007fe334d53000)
        libXtst.so.6 => /usr/lib/x86_64-linux-gnu/libXtst.so.6 (0x00007fe334b4d000)
        libatk-1.0.so.0 => not found
        libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007fe334942000)
        libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fe3346f5000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe3344d6000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fe3342a4000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe33408c000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe333c9b000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fe33c594000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fe333a93000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fe333821000)
        libpng16.so.16 => /usr/lib/x86_64-linux-gnu/libpng16.so.16 (0x00007fe3335ef000)
        libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fe3333c7000)
        libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007fe333143000)
        libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007fe332f3f000)
        libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fe332d39000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fe332b13000)
        liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fe3328f7000)
        libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007fe3325dc000)
        libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fe3323c7000)
        libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fe3321b2000)

Hope it helps

adamrehn commented 4 years ago

Hey, sorry for the delay in getting this resolved. Yes, I know from previous experience that CEF requires a number of X11-related libraries in order to run under Linux. The list of required packages under Ubuntu 18.04 is as follows:

The requirement for additional dependency packages under Linux is what lead me to suspect that the problem under Windows might be missing DLL dependencies, and running libcef.dll through Dependency Walker on my host system does indeed reveal a number of required DLL files that are not currently present in the ue4-docker Windows container images:

I'll have to look into the best way to source each of these DLLs and determine whether it's feasible to incorporate them into the ue4-build-prerequisites image. In the meantime, if you're using images based on Windows Server 2019 then you should be able to get your CI pipeline up and running by simply copying the relevant files from the mcr.microsoft.com/windows:1809 base image into your CI image.

galeone commented 4 years ago

So we are on the right track :+1:

Could you please show me how to copy the files from the base image to the CI image? I guess I simply have to change the Dockerfile - but I don't know how and where to find these DLLs in the base image filesystem.

Thank you very much for your reply!

adamrehn commented 4 years ago

You can use a Docker multi-stage build to create a new image that adds the DLL files on top of an existing ue4-docker image. For example, the following Dockerfile will add the DLLs to the ue4-full image:

# escape=`
FROM mcr.microsoft.com/windows:1809 AS dlls
FROM adamrehn/ue4-full:4.23.1-ltsc2019

COPY --from=dlls `
    C:\Windows\System32\dxva2.dll `
    C:\Windows\System32\mf.dll `
    C:\Windows\System32\mfplat.dll `
    C:\Windows\System32\mfreadwrite.dll `
    C:\Windows\System32\bthprops.cpl `
    C:\Windows\System32\BluetoothApis.dll `
    C:\Windows\System32\

However, my testing is showing that libcef.dll is still failing to load even with these files present, due to an issue loading mscoree.dll, which indicates a problem with the .NET Framework. I'll keep you posted as I investigate further.

adamrehn commented 4 years ago

Strike that last bit about the .NET issue, it appears my diagnostic tool was misbehaving. In theory those DLL files should be sufficient, although I've not yet had a chance to test it inside the ue4-full image with a project that uses a WebBrowserWidget.

galeone commented 4 years ago

I created the docker image using multi-stage build (using you Dockerfile, but inheriting from my docker image of the engine of course) and there is something weird going on: as before, I can compile successfully the game, but when I try to run the tests or to package it, everything stops working. Nothing happens after I execute the command ue4 tests --all - the CI simply goes timeout since more than 1h elapsed without doing nothing. I can't really explain what's happening

adamrehn commented 4 years ago

I've just built an image using the exact Dockerfile code provided above and performed some tests with a project that uses a WebBrowserWidget. Both the ue4 package and ue4 test --all commands work perfectly, loading the CEF library successfully and even printing log output from Chromium:

[1118/181955.572:ERROR:network_change_notifier_win.cc(156)] WSALookupServiceBegin failed with: 1722
[1118/181955.591:ERROR:audio_device_listener_win.cc(45)] RegisterEndpointNotificationCallback failed: 80070424

Evidently the DLL files listed above are indeed all that is required when extending the Windows version of the adamrehn/ue4-full:4.23.1 image. The continued presence of the loading error in your CI runs suggests that a difference in configuration must be a contributing factor. What version of the Unreal Engine are you using and in what way have you modified the ue4-full image for your CI image?

galeone commented 4 years ago

I'm using version 4.23.1 with some small changes in the engine code (mostly related to computer graphics stuff).

I just copy-pasted your Dockerfile (https://github.com/adamrehn/ue4-docker/issues/57#issuecomment-553692649), but since it works correctly in your tests, is guaranteed that the problem is somewhere in my configuration.

However, if you're going to merge these changes (fixing the CEF missing dependencies on both windows and Linux images) I guess this issue can be closed since the problem is somewhere in my codebase (pretty sure the problem is in the tests right now, I have a guess).

adamrehn commented 4 years ago

I still need to look into whether those DLL files can be sourced from anywhere other than the mcr.microsoft.com/windows base image, since that image has only existed since Windows Server 2019 / Windows 10 version 1809 and introducing a dependency on it would break compatibility with older versions of Windows. I'll keep this issue open until a solution has been implemented.

slonopotamus commented 3 years ago

I've created #165 that is explicitly dedicated to getting rid of copying DLLs from host system.

slonopotamus commented 3 years ago

This needs to be retested with modern ue4-build-prerequisites. If needed, we now can easily add libs from full windows image.

slonopotamus commented 3 years ago

Dependencies of libcef.dll as of 4.27.0:

image

It's funny that they need both d3d11.dll and d3d9.dll. DX9 in 2021, yeah, why not.

slonopotamus commented 3 years ago

Out of this list, we're still missing as of today:

bthprops.cpl
BluetoothApis.dll