UnrealBuildTool isn't built by `ue4 build`

[TBBle]

Switching UE4 branches, I cleaned my source checkout hard, using git clean -f -x d.

I then ran the Setup.bat which installed Git dependencies and pre-reqs without issue.

However, ue4 build was not able to build my project in this state.

I ran ue4 gen -vscode, and after that ue4 build worked.

This is because ue4 gen calls GenerateProjectFiles.bat on Windows which explicitly compiles UBT.

However, this was a surprising dependency to me, and there's nothing logged by the batch file that it's building UBT, so I had to check the code to make sure that this was why it worked as a fix.

I'm not sure if there's a nicer way to bootstrap UBT than used in GenerateProjectFiles.bat, and I'm not sure if this is Windows-specific, as I only tested Windows.

This isn't a problem for installed builds, and now that I know the trick, I'm not blocked on this, nor is the workaround particularly onerous.

Logs

(venv) PS D:\project\repos\unreal> ue4 build
Using user-specified engine root: D:\Unreal\UnrealEngine
Ensuring ShaderCompileWorker is built before building project Editor modules...
UnrealBuildTool.exe not found in ..\..\Engine\Binaries\DotNET\UnrealBuildTool.exe
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\Unreal\venv\Scripts\ue4.exe\__main__.py", line 7, in <module>
  File "d:\unreal\venv\lib\site-packages\ue4cli\cli.py", line 216, in main
    SUPPORTED_COMMANDS[command]['action'](manager, args)
  File "d:\unreal\venv\lib\site-packages\ue4cli\cli.py", line 70, in <lambda>
    'action': lambda m, args: m.buildDescriptor(os.getcwd(), args.pop(0) if (len(args) > 0 and args[0].startswith('-') == False) else 'Development', args.pop(0) if (len(args) > 0 and args[0].startswith('-') == False) else 'Editor',
 args),
  File "d:\unreal\venv\lib\site-packages\ue4cli\UnrealManagerBase.py", line 342, in buildDescriptor
    self._runUnrealBuildTool('ShaderCompileWorker', self.getPlatformIdentifier(), 'Development', [], capture=suppressOutput)
  File "d:\unreal\venv\lib\site-packages\ue4cli\UnrealManagerBase.py", line 657, in _runUnrealBuildTool
    Utility.run(arguments, cwd=self.getEngineRoot(), raiseOnError=True)
  File "d:\unreal\venv\lib\site-packages\ue4cli\Utility.py", line 145, in run
    raise Exception('child process ' + str(command) + ' failed with exit code ' + str(returncode))
Exception: child process ['D:\\Unreal\\UnrealEngine\\Engine\\Build\\BatchFiles\\Build.bat', 'ShaderCompileWorker', 'Win64', 'Development'] failed with exit code 999
(venv) PS D:\project\repos\unreal> ue4 gen -vscode
Using user-specified engine root: D:\Unreal\UnrealEngine
D:\project\repos\unreal\Demo.uproject
Setting up Unreal Engine 4 project files...
Discovering modules, targets and source code for project...
Binding IntelliSense data... 100%
Writing project files... 100%
(venv) PS D:\project\repos\unreal> ue4 build
Using user-specified engine root: D:\Unreal\UnrealEngine
Ensuring ShaderCompileWorker is built before building project Editor modules...
Using 'git status' to determine working set for adaptive non-unity build (D:\Unreal\UnrealEngine).
Creating makefile for ShaderCompileWorker (no existing makefile)
Creating makefile for UnrealHeaderTool (no existing makefile)
Building UnrealHeaderTool...

[adamrehn]

Having a look at Build.bat and the Linux/Mac versions of Build.sh, it looks like only the Linux version of Build.sh builds UnrealBuildTool if it isn't present. Looking through the available scripts, it appears that generating project files is the only way to build UBT under Windows and Mac (even UAT doesn't build it automatically outside of creating an Installed Build), which rules out the possibility of adding that logic to the ue4 build command.

I'm going to continue recommending that anyone using a source build of the Unreal Engine follows Epic's official instructions, which means running the top-level setup and project file generation scripts after checking out the source tree.

[TBBle]

I feel like this is something that should be addressed in the UE codebase, personally. If they're doing it on one platform, they ought to do it on all of them. Requiring GenerateProjectFiles is just a weird detour, given that they don't use those in the build at all.