Open jlsalmon opened 1 year ago
I am also running into this. I had a look and mine is using the source
image which does not contain a precompiled DLL with that name. Changing to minimal
doesn't work because generate cannot use an installed build of UE apparently.
I hacked in a copy from the minimal
image of the UnrealBuildTool
directory and now the next error is:
#14 [conan 6/6] RUN ue4 conan generate && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that
for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo ''
#14 sha256:9586126ecafb49460afac2b01fb0b22e089cbbf4b1a05709303b7dda46438cb5
#14 0.457 Using user-specified engine root: /home/ue4/UnrealEngine
#14 6.792 Traceback (most recent call last):
#14 6.792 File "/usr/local/bin/ue4", line 8, in <module>
#14 6.792 sys.exit(main())
#14 6.792 File "/usr/local/lib/python3.10/dist-packages/ue4cli/cli.py", line 222, in main
#14 6.792 Removing the "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile if it already exists... #14 6.792 Removing any previous versions of profile base packages...
#14 6.792 Creating "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile using autodetected settings...
#14 6.792 Installing profile base packages...
#14 6.792 Copying the "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile into a new profile named "ue4"...
#14 6.792 Retrieving thirdparty library list from UBT...
#14 6.792 Removing any previous versions of generated wrapper packages for 5.1... #14 6.792 Generating and installing toolchain wrapper package...
#14 6.792 Wrapping clang: /home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v20_clang-13.0.1-centos7/x86_64-unknown-linux-gnu/
bin/clang
#14 6.792 Wrapping lib++: /home/ue4/UnrealEngine/Engine/Source/ThirdParty/Unix/LibCxx/lib/Unix/x86_64-unknown-linux-gnu/libc++.a
#14 6.792 SUPPORTED_COMMANDS[command]['action'](manager, args) #14 6.792 File "/usr/local/lib/python3.10/dist-packages/conan_ue4cli/main.py", line 48, in main
#14 6.792 SUBCOMMANDS[subcommand]['function'](manager, args[1:])
#14 6.792 File "/usr/local/lib/python3.10/dist-packages/conan_ue4cli/commands/generate.py", line 169, in generate
#14 6.792 PackageManagement.install(join(packagesDir, 'toolchain-wrapper'), channel, profile, [
#14 6.792 File "/usr/local/lib/python3.10/dist-packages/conan_ue4cli/common/PackageManagement.py", line 16, in install
#14 6.792 return Utility.run(['conan', 'create', '.', 'adamrehn/' + channel, '--profile=' + profile] + args, cwd=packageDir)
#14 6.792 File "/usr/local/lib/python3.10/dist-packages/conan_ue4cli/common/Utility.py", line 29, in run
#14 6.792 raise Exception(
#14 6.792 Exception: child process ['conan', 'create', '.', 'adamrehn/5.1', '--profile=ue5.1-Linux-x86_64-unknown-linux-gnu', '--env', 'WRAPPED_TOOLCHAIN=/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v20_clang-13.0.1-centos7/x86_64-unknown-linux-gnu', '--env', 'WRAPPED_LIBCXX=/ho
me/ue4/UnrealEngine/Engine/Source/ThirdParty/Unix/LibCxx'] failed with exit code 1
#14 6.792 stdout: "Exporting package recipe #14 6.792 toolchain-wrapper/ue4@adamrehn/5.1 exports: Copied 4 '.py' files: conanfile.py, clang.py, clang++.py, ClangInterposition.py #14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: A new conanfile.py version was exported
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Folder: /home/ue4/.conan/data/toolchain-wrapper/ue4/adamrehn/5.1/export
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Exported revision: 3845df134b0b0b303c03d66020e6c06c #14 6.792 Configuration: #14 6.792 [settings]
#14 6.792 arch=x86_64
#14 6.792 arch_build=x86_64
#14 6.792 build_type=Release
#14 6.792 compiler=clang #14 6.792 compiler.libcxx=libc++
#14 6.792 compiler.version=13
#14 6.792 os=Linux #14 6.792 os_build=Linux
#14 6.792 [options]
#14 6.792 [build_requires]
#14 6.792 *: toolchain-wrapper/ue4@adamrehn/5.1
#14 6.792 [env]
#14 6.792 UNREAL_ENGINE_VERSION=5.1
#14 6.792 WRAPPED_LIBCXX=/home/ue4/UnrealEngine/Engine/Source/ThirdParty/Unix/LibCxx
#14 6.792 WRAPPED_TOOLCHAIN=/home/ue4/UnrealEngine/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v20_clang-13.0.1-centos7/x86_64-unknown-linux-gnu
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Forced build from source
#14 6.792 Installing package: toolchain-wrapper/ue4@adamrehn/5.1
#14 6.792 Requirements
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1 from local cache - Cache
#14 6.792 Packages
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1:7398dab5728f06866bd82627d2e411754f86297a - Build
#14 6.792
#14 6.792 Installing (downloading, building) binaries...
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Configuring sources in /home/ue4/.conan/data/toolchain-wrapper/ue4/adamrehn/5.1/source
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Copying sources to build folder
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Building your package in /home/ue4/.conan/data/toolchain-wrapper/ue4/adamrehn/5.1/build/7398dab5728f06866bd826
27d2e411754f86297a
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Generator txt created conanbuildinfo.txt
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Aggregating env generators
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Calling build()
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Package '7398dab5728f06866bd82627d2e411754f86297a' built
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Build folder /home/ue4/.conan/data/toolchain-wrapper/ue4/adamrehn/5.1/build/7398dab5728f06866bd82627d2e411754f
86297a
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Generated conaninfo.txt
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Generated conanbuildinfo.txt
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Generating the package
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Package folder /home/ue4/.conan/data/toolchain-wrapper/ue4/adamrehn/5.1/package/7398dab5728f06866bd82627d2e411
754f86297a
#14 6.792 toolchain-wrapper/ue4@adamrehn/5.1: Calling package()
#14 6.792 "
#14 6.792 stderr: "toolchain-wrapper/ue4@adamrehn/5.1: WARN: This conanfile has no build step
#14 6.792 ERROR: toolchain-wrapper/ue4@adamrehn/5.1: Error in package() method, line 69
#14 6.792 libraries = dirname(self._find_libcxx(libcxx, architecture))
#14 6.792 while calling '_find_libcxx', line 42
#14 6.792 raise RuntimeError('Failed to locate libc++.a for architecture "{}" inside directory "{}"!'.format(architecture, root))
#14 6.792 RuntimeError: Failed to locate libc++.a for architecture "x86_64" inside directory "/home/ue4/UnrealEngine/Engine/Source/ThirdParty/Unix/Lib
Cxx"!
Quick check finds this:
ue4@895cb72076f3:~/UnrealEngine/Engine/Source/ThirdParty$ find . | grep libc++.a
./Unix/LibCxx/lib/Unix/aarch64-unknown-linux-gnueabi/libc++.a
./Unix/LibCxx/lib/Unix/x86_64-unknown-linux-gnu/libc++.a
While looking in the Unreal 4.27.2 image I found this:
ue4@abee626cafda:~/UnrealEngine/Engine/Source/ThirdParty$ find . | grep libc++.a
./Linux/LibCxx/lib/Linux/aarch64-unknown-linux-gnueabi/libc++.a
./Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++.a
Looks like the file names have changed in this toolchain version, sigh.
What would be the best way to incorporate the fix in https://github.com/adamrehn/conan-ue4cli/pull/21 ? Unfortunately it is not in a pip release yet, so just using ue4-docker installs an old version without that fix. This currently prevents me from building UE 5.0.3 or 5.1.
I haven't tested that fix, but it would be straightforward to monkey patch conan-ue4cli in the absence of this being released.
For a local conan-ue4cli I agree, but I am actually only using ue4-docker where it builds some Dockerfile using it. Not sure how the inner workings of ue4-docker are.
I think I modified the Dockerfile part where it pip installed it to pip install https://github.com/Greenroom-Robotics/conan-ue4cli/archive/refs/heads/feature/fix-libcxx.zip
According to this https://github.com/adamrehn/ue4-docker/blob/c4e8ecec9b0f1dfdee0d6fa96cd4592c6a016644/ue4docker/infrastructure/BuildConfiguration.py#L225
You could pass an argument to ue4-docker: -conan-ue4cli=https://github.com/Greenroom-Robotics/conan-ue4cli/archive/refs/heads/feature/fix-libcxx.zip
.
You could pass an argument to ue4-docker:
-conan-ue4cli=https://github.com/Greenroom-Robotics/conan-ue4cli/archive/refs/heads/feature/fix-libcxx.zip
.
I tried that (earlier I tried these two patches as well) - but still get this error (both on 5.0.3 and 5.1.0):
----
> [conan 5/5] RUN ue4 conan generate && echo '' && echo 'RUN directive complete. Docker will now commit the filesystem layer to disk.' && echo 'Note that for large filesystem layers this can take quite some time.' && echo 'Performing filesystem layer commit...' && echo '':
#13 4.305 Removing the "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile if it already exists...
#13 4.305 Removing any previous versions of profile base packages...
#13 4.305 Creating "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile using autodetected settings...
#13 4.305 Installing profile base packages...
#13 4.305 Copying the "ue5.1-Linux-x86_64-unknown-linux-gnu" Conan profile into a new profile named "ue4"...
#13 4.305 Retrieving thirdparty library list from UBT...
#13 4.305 Using user-specified engine root: /home/ue4/UnrealEngine
#13 4.306 Traceback (most recent call last):
#13 4.306 File "/usr/local/bin/ue4", line 8, in <module>
#13 4.306 sys.exit(main())
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/cli.py", line 222, in main
#13 4.306 SUPPORTED_COMMANDS[command]['action'](manager, args)
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/conan_ue4cli/main.py", line 48, in main
#13 4.306 SUBCOMMANDS[subcommand]['function'](manager, args[1:])
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/conan_ue4cli/commands/generate.py", line 153, in generate
#13 4.306 libs = [lib for lib in manager.listThirdPartyLibs() if lib != 'libc++']
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/UnrealManagerBase.py", line 183, in listThirdPartyLibs
#13 4.306 return interrogator.list(self.getPlatformIdentifier(), configuration, self._getLibraryOverrides())
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/UE4BuildInterrogator.py", line 20, in list
#13 4.306 modules = self._getThirdPartyLibs(platformIdentifier, configuration)
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/UE4BuildInterrogator.py", line 170, in _getThirdPartyLibs
#13 4.306 self.runUBTFunc('UnrealEditor', platformIdentifier, configuration, args)
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/UnrealManagerBase.py", line 695, in <lambda>
#13 4.306 ubtLambda = lambda target, platform, config, args: self._runUnrealBuildTool(target, platform, config, args, True)
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/UnrealManagerBase.py", line 687, in _runUnrealBuildTool
#13 4.306 return Utility.capture(arguments, cwd=self.getEngineRoot(), raiseOnError=True)
#13 4.306 File "/usr/local/lib/python3.8/dist-packages/ue4cli/Utility.py", line 125, in capture
#13 4.306 raise Exception(
#13 4.306 Exception: child process ['/home/ue4/UnrealEngine/Engine/Build/BatchFiles/Linux/Build.sh', 'UnrealEditor', 'Linux', 'Development', '-Mode=JsonExport', '-OutputFile=/tmp/tmpsisfjm1z/ubt_output.json'] failed with exit code 1
#13 4.306 stdout: "Setting up bundled DotNet SDK
#13 4.306 Running command : dotnet Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll UnrealEditor Linux Development -Mode=JsonExport -OutputFile=/tmp/tmpsisfjm1z/ubt_output.json
#13 4.306
#13 4.306 Welcome to .NET !
#13 4.306 ---------------------
#13 4.306 SDK Version:
#13 4.306
#13 4.306 Telemetry
#13 4.306 ---------
#13 4.306 The .NET tools collect usage data in order to help us improve your experience. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.
#13 4.306
#13 4.306 Read more about .NET CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry
#13 4.306
#13 4.306 ----------------
#13 4.306 Installed an ASP.NET Core HTTPS development certificate.
#13 4.306 To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
#13 4.306 Learn about HTTPS: https://aka.ms/dotnet-https
#13 4.306 ----------------
#13 4.306 Write your first app: https://aka.ms/dotnet-hello-world
#13 4.306 Find out what's new: https://aka.ms/dotnet-whats-new
#13 4.306 Explore documentation: https://aka.ms/dotnet-docs
#13 4.306 Report issues and find source on GitHub: https://github.com/dotnet/core
#13 4.306 Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
#13 4.306 --------------------------------------------------------------------------------------
#13 4.306 "
#13 4.306 stderr: "Could not execute because the specified command or file was not found.
#13 4.306 Possible reasons for this include:
#13 4.306 * You misspelled a built-in dotnet command.
#13 4.306 * You intended to execute a .NET program, but dotnet-Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll does not exist.
#13 4.306 * You intended to run a global tool, but a dotnet-prefixed executable with this name could not be found on the PATH.
#13 4.306 "
----
I came across this as well. Engine/Binaries/DotNET/UnrealBuildTool/UnrealBuildTool.dll
is not compiled in that image. It's all a bit of a mess. I had to copy the compiled UnrealBuildTool from the minimal
image.
You can see what I did here in this commit https://github.com/Greenroom-Robotics/ue4-docker/commit/04a48e4125bd5c1363c298c1b0214439f60592e4
You will have to change it to reflect your image names.
thanks, that worked :-)
(Moved from https://github.com/adamrehn/ue4-docker/issues/281)
Output of the
ue4-docker info
command:Building 5.1.0 fails at the
ue4 conan generate
step:Could this have something to do with UBT needing to be pre-built?