Upgrade 4.22

[meiwen333]

Hi,

I know this is not particular a bug. But in 4.22 there are quiete a few changes in the UBT which are not yet in FastBuild. Some classes have changed completely. I will have a look into the code but it seems quite a bit of work which I am not sure I can handle on my own. Anybody worked through it so far?

[meiwen333]

I got some stuff fixed. Like the path that needs to be converted to a string. But some of the classes in the buildpipline have changed completely. So the sorting of actions do not work anymore. Unfortunately I am not that familiar with Fastbuild to get my head around that any time soon. Is there a better documentation that could point me in the right direction?

[wout276]

Hi @meiwen333 I've stumbled upon the same issue as you. For the path, I simply changed action.CommandPath.Contains() to action.CommandPath.FullName.Contains(). As far as I'm aware there are no other useful docs for this but the idea is pretty simple. Fastbuild distributes all actions but most actions require items produced by other actions as prerequisite. I'm looking for another way to get those ProducingActions as well. The best source for me is the XGE executor (which is from Incredibuild) in UnrealBuildTool/Executors and the changes to their workflow between 4.21 and 4.22.

I'll let you know if I make any progress.

[MrPatry]

We got it to work with some small changes (one of which (the regex) is really hacky). You're welcome to try it. It worked for us but I can't guarantee it's 100% error-free

(Update 12:17PM: New version attached, some error happened when compiling c files in LiveCoding)

FASTBuild.zip

[meiwen333]

Hi @MrPatry ,

thanks for the update. Did you also experience all the missing precrocessor macros? Seems similar to this thread: https://answers.unrealengine.com/questions/813258/updated-to-420-now-i-cant-build-error-c4668.html

Anyway thanks for posting the solution to the upgrade :)

Cheers Pascal

[MrPatry]

No we did not, sorry

[yinweihn]

We got it to work with some small changes (one of which (the regex) is really hacky). You're welcome to try it. It worked for us but I can't guarantee it's 100% error-free

(Update 12:17PM: New version attached, some error happened when compiling c files in LiveCoding)

FASTBuild.zip

Hi @MrPatry I use your code in vs2017 but error occured error described that: opening file 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.13.26020\x64\Microsoft.VC141.CRT\msvcp140.dll' in Compiler ToolManifest

[MrPatry]

Hi @MrPatry I use your code in vs2017 but error occured error described that: opening file 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.13.26020\x64\Microsoft.VC141.CRT\msvcp140.dll' in Compiler ToolManifest

Make sure you're using at least VS2017 15.9 and have .Net Framework 4.6.2 targetting pack installed

[meiwen333]

In my case I changed the path to the correct version: C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Redist\MSVC\14.16.27012\x64\Microsoft.VC141.CRT It is in line 617-618 or 624-625

if (VCEnv.Compiler == WindowsCompiler.VisualStudio2015)
                {
                    AddText(string.Format("\t\t'{0}/VC/redist/x64/Microsoft.VC{1}.CRT/msvcp{2}.dll'\n", VCInstallDir.ToString(), platformVersionNumber, platformVersionNumber));
                    AddText(string.Format("\t\t'{0}/VC/redist/x64/Microsoft.VC{1}.CRT/vccorlib{2}.dll'\n", VCInstallDir.ToString(), platformVersionNumber, platformVersionNumber));
                }
                else
                {
                    //VS 2017 is really confusing in terms of version numbers and paths so these values might need to be modified depending on what version of the tool chain you
                    // chose to install.
                    AddText(string.Format("\t\t'{0}/VC/Redist/MSVC/14.13.26020/x64/Microsoft.VC141.CRT/msvcp{1}.dll'\n", VCInstallDir.ToString(), platformVersionNumber));
                    AddText(string.Format("\t\t'{0}/VC/Redist/MSVC/14.13.26020/x64/Microsoft.VC141.CRT/vccorlib{1}.dll'\n", VCInstallDir.ToString(), platformVersionNumber));
                }

Just check which version you have and adjust the path.

[meiwen333]

Anyboday experiencing these errors: C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared\intsafe.h(66): error C4668: '__midl' is not defined as a preprocessor macro, replacing with '0' for '#if/#elif'

I just found this post: https://answers.unrealengine.com/questions/856910/error-c4668-cplusplus-not-defined.html?sort=oldest

And it seems that the win sdk is the culprit. Which version are you using @MrPatry ?

[wout276]

Thx @MrPatry ! My own fix for the command-line parsing issue was a lot less clean/more hacky.. The other small changes you made were what was missing on my end. I'm just hitting one small issue: just before the build completely finishes, this happens:

1>5> -Build: 351 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-DerivedDataCache.dll
1>9> -Build: 1158 ms    D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-RHI.dll
1>11>-Build: 381 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Json.dll
1>12>-Build: 638 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ApplicationCore.dll
1>8> -Build: 535 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Projects.dll
1>7> -Build: 451 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ShaderCompilerCommon.dll
1>10>-Build: 423 ms D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-InputCore.dll
1>1> -Build: 1494 ms    D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-VulkanShaderFormat.dll
1>3> -Build: 3792 ms    D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Core.dll
1>2> -Build: 3429 ms    D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Slate.dll
1>4> -Build: 4265 ms    D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-CoreUObject.dll
1>6> -Build: 113871 ms  D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\UE4Editor-Engine.dll
1>Alias: all
1>Failed due to missing file: D:\GitHub\UE4.22\UnrealEngine\Engine\Source\Action_1744
1>Saving DepGraph 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\fbuild.fdb'
1>Saving DepGraph Complete in 0.068s

I looked a further into this issue and saw that this actions are not explicitly created in the bff file, just it's command line command is in there:

// "D:\GitHub\UE4.22\UnrealEngine\Engine\Build\Windows\cl-filter\cl-filter.exe" D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding\xxhash.c.txt -- "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\cl.exe"  @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding\xxhash.c.obj.response" /showIncludes
ObjectList('Action_1743')
{
    .Compiler = 'UE4Compiler' 
    .CompilerInputFiles = "D:\GitHub\UE4.22\UnrealEngine\Engine\Source\Developer\Windows\LiveCoding\Private\External\xxhash.c"
    .CompilerOutputPath = "D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\UE4Editor\Development\LiveCoding"
    .CompilerOptions = '/Zc:inline /nologo /Oi /c /Gw /Gy /Zm850 /wd4819 /D_CRT_STDIO_LEGACY_WIDE_SPECIFIERS=1 /D_SILENCE_STDEXT_HASH_DEPRECATION_WARNINGS=1 /D_WINDLL /D_DISABLE_EXTENDED_ALIGNED_STORAGE /source-charset:utf-8 /execution-charset:utf-8 /Ox /Ot /GF /Ob2 /errorReport:prompt /EHsc /Z7 /MD /bigobj /fp:fast /Zo /Zp8 /wd4463 /wd4838 /I . /I Developer\Windows\LiveCoding\Private /I Runtime /I Runtime\Core\Public /I ..\Intermediate\Build\Win64\UE4Editor\Inc\CoreUObject /I Runtime\CoreUObject\Public /I Developer /I Developer\Settings\Public /I ..\Intermediate\Build\Win64\UE4Editor\Inc\LiveCoding /I Developer\Windows /I Developer\Windows\LiveCoding\Public /I "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\INCLUDE" /I "C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um" /I "C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt" /DIS_PROGRAM=0 /DUE_EDITOR=1 /DENABLE_PGO_PROFILE=0 /DUSE_VORBIS_FOR_STREAMING=1 /DWITH_DEV_AUTOMATION_TESTS=1 /DWITH_PERF_AUTOMATION_TESTS=1 /DUNICODE /D_UNICODE /D__UNREAL__ /DIS_MONOLITHIC=0 /DWITH_ENGINE=1 /DWITH_UNREAL_DEVELOPER_TOOLS=1 /DWITH_APPLICATION_CORE=1 /DWITH_COREUOBJECT=1 /DUSE_STATS_WITHOUT_ENGINE=0 /DWITH_PLUGIN_SUPPORT=0 /DWITH_PERFCOUNTERS=1 /DUSE_LOGGING_IN_SHIPPING=0 /DWITH_LOGGING_TO_MEMORY=0 /DUSE_CACHE_FREED_OS_ALLOCS=1 /DUSE_CHECKS_IN_SHIPPING=0 /DWITH_EDITOR=1 /DWITH_SERVER_CODE=1 /DWITH_CEF3=1 /DWITH_LIVE_CODING=1 /DWITH_XGE_CONTROLLER=1 /DUBT_MODULE_MANIFEST=\"UE4Editor.modules\" /DUBT_MODULE_MANIFEST_DEBUGGAME=\"UE4Editor-Win64-DebugGame.modules\" /DUBT_COMPILED_PLATFORM=Win64 /DUBT_COMPILED_TARGET=Editor /DUE_APP_NAME=\"UE4Editor\" /DWIN32=1 /D_WIN32_WINNT=0x0601 /DWINVER=0x0601 /DPLATFORM_WINDOWS=1 /DNDEBUG=1 /DUE_BUILD_DEVELOPMENT=1 /DUE_IS_ENGINE_MODULE=1 /DUE_ENABLE_ICU=1 /DWITH_VS_PERF_PROFILER=0 /DWITH_DIRECTXMATH=0 /DWITH_MALLOC_STOMP=1 /DCORE_API=DLLIMPORT /DCOREUOBJECT_API=DLLIMPORT /DSETTINGS_API=DLLIMPORT /DLIVECODING_API=DLLEXPORT /TC /W0  /Fo"%2" "%1" '
    .CompilerOutputExtension = '.c.obj' 
}

// "C:\WINDOWS\system32\cmd.exe" /C "copy /Y "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\ThirdParty\Windows\DirectX\x64\dxcompiler.dll" "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\dxcompiler.dll" 1>nul"
// "C:\WINDOWS\system32\cmd.exe" /C "copy /Y "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\ThirdParty\Windows\DirectX\x64\dxil.dll" "D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\dxil.dll" 1>nul"
// "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.exe" @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response"
Copy('Action_1746_dummy')
{ 
    .Source = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response' 
    .Dest = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response.dummy' 
    .PreBuildDependencies = {
        'Action_200', ;SharedPCH.Core.NoUndef.cpp
        'Action_1108', ;Module.ShaderFormatD3D.cpp
        'Action_278', ;ShaderCompileWorker-Core.lib
        'Action_534', ;ShaderCompileWorker-RenderCore.lib
        'Action_763', ;ShaderCompileWorker-ShaderPreprocessor.lib
        'Action_765', ;ShaderCompileWorker-ShaderCompilerCommon.lib
    } 
}

Executable('Action_1746')
{ 
    .Linker = 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64\link.exe' 
    .Libraries = 'Action_1746_dummy' 
    .LinkerOptions = '/TLBOUT:"%1" /Out:"%2" @"D:\GitHub\UE4.22\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\ShaderFormatD3D\ShaderCompileWorker-ShaderFormatD3D.dll.response" ' 
    .LinkerOutput = 'D:\GitHub\UE4.22\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-ShaderFormatD3D.dll' 
}

This action, along with some other actions (like 1745 in my case - as you can also see in the snippet above) are missing as well. Checking the ActionTypes, all of them are BuildProject or WriteMetaData steps. This is as expected from CreateBffFile(). Since this function hasn't really changed in comparison to our working setups for earlier engine versions, I'm in the dark about why these actions have suddenly become a problem.

[MrPatry]

@wout276 Try with this new version, it should fix this problem

FASTBuild.zip

[wout276]

@MrPatry Thx again for the quick answer. I just figured it out myself and coded it slightly differently. Here's my solution - with what I had to change to make caching work as well.

Edit 17:20 fixed cashing issue: FASTBuild.zip

[MrPatry]

And it seems that the win sdk is the culprit. Which version are you using @MrPatry ?

@meiwen333 I'm using 10.0.16299.0

[msolemajor]

hello @meiwen333 / @wout276 with all this setup do you have buld failure, that are complaining of missing Lib files... ?

11>UnrealBuildTool : error : Failed to produce item: ....\Unreal-Stable\Engine\Plugins\Tests\RuntimeTests\Intermediate\Build\Win64\UE4Editor\Development\RuntimeTests\UE4Editor-RuntimeTests.lib

those kind?

[meiwen333]

Hi, @msolemajor I didn't have these errors.

I got it up and running with a change that was suggested in this thread: https://github.com/liamkf/Unreal_FASTBuild/issues/29

[MrPatry]

Hey guys, just to let you know that instead of having to do the regex trick, you can simply set bGenerateDependenciesFile to false in CppCompileEnvironment.cs

It will also allow incremental builds to work.

Also, I suggest adding this to your BuildConfiguration.xml file to prevent lib linking issues

<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
    <BuildConfiguration>
        <bEditorDependsOnShaderCompileWorker>false</bEditorDependsOnShaderCompileWorker>
    </BuildConfiguration>
</Configuration>

Cheers

[oliviergaertner]

Hey Guys! I've integrated @MrPatry file but I have a weird behavior: every time I try to build the project it looks like the previous actions listed are rebuilt again and again. ex: I launch a build and only have one cpp to build. The build complete as expected. If I launch the build again the same cpp is built again without any changes. Does it ring a bell to anyone? Cheers

[wout276]

@oliviergaertner I think this was fixed with @MrPatry latest comment allowing incremental builds to work:

Hey guys, just to let you know that instead of having to do the regex trick, you can simply set bGenerateDependenciesFile to false in CppCompileEnvironment.cs

It will also allow incremental builds to work.

[oliviergaertner]

According to some threads on UDN setting bGenerateDependenciesFile to false might generate some linking issues down the road. Are experiencing any? Like when adding UFunctions etc?

[nullbus]

IMO setting bGenerateDependenciesFile to false should does the trick, but as it doesn't generate dependencies file so it cannot does incremental build properly. After I turned it to false and I had to rebuild all files every time.

There should be a way to generate dependencies file with FastBuild instead of cl-filter.exe.

[nucleiis]

@oliviergaertner @nullbus After merging @MrPatry's work, I am experiencing full rebuild every time. Is this expected behavior at this moment with FBuild on 4.22?

Anyone tried putting 4.21's UBT back to 4.22?

[nullbus]

I made some fixes to generate dependency files with FastBuild and got it working in 4.22 Windows Editor(with VS2019). As it adds more pass so the compilation time has increased. And if you are building engine, there are still some issues that can be fixed or detoured manually:

• I encountered C1001 compiler error when compiler tries to pass #import directive in the engine source code. If you look at the errorous source code, you might know these are easily turned off setting macro variables.
• Use English as the primary language pack of the Visual Studio. I used Korean(my mother tongue) at first, but it generated empty dependency files.

... And also I think there are some possible improvements:

• Distribute dependency generation as it is almost same as normal compile process.
• ....

I attached my FASTBuild.cs file here but it differs much from original code to fit my usage. So please use at your own risk.

FASTBuild.zip

[ZiggyCarveth]

I made some fixes to generate dependency files with FastBuild and got it working in 4.22 Windows Editor(with VS2019). As it adds more pass so the compilation time has increased. And if you are building engine, there are still some issues that can be fixed or detoured manually:

• I encountered C1001 compiler error when compiler tries to pass #import directive in the engine source code. If you look at the errorous source code, you might know these are easily turned off setting macro variables.
• Use English as the primary language pack of the Visual Studio. I used Korean(my mother tongue) at first, but it generated empty dependency files.

... And also I think there are some possible improvements:

• Distribute dependency generation as it is almost same as normal compile process.
• ....

I attached my FASTBuild.cs file here but it differs much from original code to fit my usage. So please use at your own risk.

FASTBuild.zip

I've tried using this and am getting this error:

2> at System.IO.Path.CheckInvalidPathChars(String path, Boolean checkAdditional) 2> at System.IO.Path.InternalGetDirectoryName(String path) 2> at UnrealBuildTool.FASTBuild.AddCompileAction(Action Action, Int32 ActionIndex, List1 DependencyIndices) in C:\Program Files\Epic Games\UE_4.22\Engine\Source\Programs\UnrealBuildTool\System\FASTBuild.cs:line 854 2> at UnrealBuildTool.FASTBuild.CreateBffFile(List1 InActions, String BffFilePath, List1 LocalExecutorActions) in C:\Program Files\Epic Games\UE_4.22\Engine\Source\Programs\UnrealBuildTool\System\FASTBuild.cs:line 1220`

Not sure whats wrong, might be due to an update since this is a few months old. Hopefully someone smarter than me can figure out how to make this work with 4.22

[wout276]

Not sure if related, but I got an error in AddCompileAction on line 958. It was due to the LastIndexOf command which in some cases returned -1. Adding a second >0 check solved that error for me.

I merged the fixes made by @nullbus with our own changes and it worked perfectly on the latest 4.22 so I don't think the version is the issue in your case.

[ZiggyCarveth]

@wout276 Hey, thanks so much for the reply and uploading your fastbuild.cs. Do you think you could upload your most recent fastbuild.cs? I got your last uploaded version working, but only after adding "bEnableUndefinedIdentifierWarnings = false;" to my build.cs which I'm certain will cause issues down the line, so I'd like to see if you did anything I'm missing.

[wout276]

@JacobShawC Setting bEnableUndefinedIdentifierWarnings = false; is unfortunate indeed, but since we don't experience many issues due to it being disabled, I have never searched for any solutions. Afaik, it's something everyone working with fastbuild since 4.20 is doing.

We recently switched to @nullbus' fastbuild.cs and doing the extra pass to generate the dependency files. However, since we are heavily using caching and the extra pass is not distributed nor cached, it feels like doing the extra pass and an incremental build is slower in our case than doing a full rebuild every time, even for small changes. Can't tell for sure yet though.

Anyway, here is our current fastbuild.cs file. It's mostly the same as @nullbus' file now but with some minor changes.

FASTBuild.zip

[nullbus]

It's been long time I have done it before. For the file I posted earlier, there is a problem that build distribution not started until generating bunch of dependency files. (For engine build, I had to wait 5+ minutes with 12 threads). I don't know why it's not starting because its dependency have already finished.

But when I reversed the dependency between 'obj file' and 'dependency txt file' in bff file, build distribution started almost immediately as I expected. If you are building with remote machine(s), it would show better performance.

I cannot share my file right now because it needs more improvement(including merging @wout276's work). But I think it's kinda easy to implement.

[wout276]

Hi @nullbus , I don't completely understand why reversing the dependency would result in an overall speedup or what exactly you mean by reversing the dependency. Changing the dependency from obj->txt to txt->obj and then passing the dependency txt's to fastbuild as targets does indeed start the build immediately. Although, in my tests, the total time is the same because the dependency files still get created afterwards. I think the only way to speed this up would be to make distribution work for the extra pass or to do everything in one pass. Shouldn't it be possible to use cl-filter (which creates a cl.exe child process) to generate the txt files as well as the object files? I've tried a little bit to get it working in one pass but I've had no luck so far.

[nullbus]

@wout276 I understand your situation, as generating dependency files are only processed in locally now. I also usually waited 1-2 more minutes for dependency files.

And yes, (I guess) cl-filter.exe's original purpose is creating a dependency file with a object file. But I wonder it also could be done with FastBuild, which cannot handle multiple output files with single ObjectList node. It is also discussed at FastBuild issue 2 years ago. IMHO distributed cl-filter pass could be possible after that issue has resolved.

And I found another issue that incremental build almost not working. The cause was about dependency file pass(second pass). To make process faster and to make dependency file without generating object file, I used /E flag that emits preprocessed source files to stdout. And this huge amount of output mixed with dependency info(/showIncludes emits information to stderr) blocks cl-filter to process properly. I tried a lot to solve that problem but I got no luck and I had to make modified cl-filter that handle stderr and stdout separately.

I made a gist which issues I mentioned above are resolved. To build cl-filter, use visual studio solution located at Engine/Extras/cl-filter and replace built binary to Engine/Build/Windows/cl-filter.

[wout276]

@nullbus Thx for sharing your optimization, I'll do some more testing and we'll go with the solution that's currently fastest for us. It's a hacky workaround but if we're not satisfied yet, I may look into archiving the output files together and referring that archive as the outputfile to fastbuild and de-archiving on the client side. It will also create overhead but with a fast archiving algorithm, it could be worth it.

[ShuraLiu]

Hi @nullbus @wout276 , I'm trying to add fastbuild into the engine these days and I successfully builded the engine with your files, Thanks for your jobs!

But I can not fix the incremantal build issue. The engine does full build every time even with no modifications, and the build takes unacceptable time althought the the fastbuild cache works.

I tried everything i can do but got no progress, any ideas?

[wout276]

Hi @ShuraLiu, with the changes made by @nullbus, a full rebuild shouldn't happen anymore. It does however rebuild all dependency files every time, even with caching enabled. Unfortunately, there's currently no solution for this. I suggest you test for your own case whether a full cached rebuild is faster than the partial rebuild with dependency files and choose the appropriate one. Unfortunately, building will in any case be slower than in UE4.21. It will probably stay this way for future engine versions as well, until Fastbuild supports multiple output files or someone finds and implements another workaround.

[wout276]

https://github.com/Ronsenval/UnrealEngineCompilationUsingFastBuild This is a new repo by @Ronsenval , I didn't test his changes yet but he might have found a nice solution for the dependency generation. It might be interesting for at least some of you.

[oliviergaertner]

I've just integrated @Ronsenval solution for 4.23 but at the moment I have some link issues (very long link time and linker memory goes crasy)

[wout276]

When I tested his solution the last few days, I had very long build times since it rebuilded big parts of the engine even though I didn't change a letter of code. I was able to setup my own version for 4.23 though by adding the Intel ISPC compiler. I guess currently it's still a matter of choosing the implementation with the downside that's the least bad for your own situation..

[FBubbles]

So is development for this now dead? We're now approaching 4.24 and still no official release for a compatible version.

I've unfortunately had to skip using fastbuild as the fixes posted here had too many issues.

[duplexsystem]

cricket cricket

[Hisamera]

As Incredibuild prices are a little too much for starting indie, and my particular refusal to quit against my better judgement I'm working on porting FASTBuild to 4.24 (but I'm just tinkering in my free time), can't promise if I'll be able to do it and even when. But as I fixed some PhysX bugs in UE4 before (NDA source, cannot publish it anywhere) I hope it won't be too hard.

[wout276]

@Hisamera The problem is not getting it to work, there's still a few people/companies using this. The problem is just that changes were made to UnrealBuildTool that aren't compitable with Fastbuild. Thus requiring workaround and losing a lot of the performance gain in comparison to before and making the development time not quite worth it. On top of that, I found some hints in the code of 4.25 and 4.26 that a native fastbuild implementation might be coming to the official UE4 releases soon(?) - with the needed BuildTool changes. Also Oculus has implemented fastbuild to some extend in it's version of UE4, but didn't really solve the need for an extra dependency pass afaik. These reasons together are why development appears to be dead currently. You can never get the performance gains you could get before without making drastic changes to the build tool or Fastbuild itself.

edit: If you want I can send you a functional version for UE4.24 but we ditched it since. It also breaks the in engine compile button in some situations and breaks hot-reload. These issues can probably be fixed though - but I didn't think it was still worth the time.

[Hisamera]

@wout276 I'm not doing it for profit (I'm working on porting FASTBuild to 4.24 (but I'm just tinkering in my free time) ). I never would have been where I currently am, if I just did things that were profitable, or worth the hassle. I fixed broken PhysX implementation in UE4, even though turning it off would have been sufficient, that should tell you a lot about my ambitions.

Secondly, I highly respect your willingness to give me your edited file, but I'm doing it precisely because it's not easy. I'm like a homing rocket when it goes for problems with C++ irrelevant of where it originated, whenether in UE4 source, or projects source, because I did things the hard way first time. The only thing that is totally opaque to me is UBT, writing a FASTBuild integration is a quickest way for me to learn it. I'm doing programming stuff for over 10 years now, recently professionally, and I never learned quicker than when I was faced with a problem.

Thirdly I looked at a lot of implementations, and attempts to fix them, but I decided to write it from scratch, and I'm nearing finishing (My file already has 40KB of code, and probably will double) local compilation (so all cores are used not just one), from my understanding caching would be rather easy to implement (as FASTBuild just does it itself), and distributed could take a long time, but company where I'm employed is trying to give me a exclusive access to one of their lesser PCs. All of that was done in my free time during a week nontheless, I'm optimistic about finishing it this weekend.

Fourthly, It won't be a wasted effort for ME, even if UE4.26 will have it natively, as I will learn a ton of stuff about UBT and other stuff.

Fifthly, thats a last thing that keep nagging me, and I want to return to writing modding tools for Stellaris as I always wanted, but I couldn't for a long time XD

[ricekab]

edit: If you want I can send you a functional version for UE4.24 but we ditched it since.

What did you ditch it for? Other than Incredibuild I'm not aware of an alternative.

[wout276]

@Hisamera I should have clarified it is still extremely useful for building local only, as it's only the distributed build step that is hindered by Fastbuild's limitations. If your implementation is finished and you're willing to share it, I would like to have a look at it out of curiosity :)

@ricekab We didn't replace it, it's still the best alternative and if we really need it again, I can have it running in a few hours. There's just some inconveniences when working from a custom binary build that's build with fastbuild on pc's that don't have the exact same setup.

[Peak2018]

Compile Unreal Engine 4.25.3 using FastBuild Failed due to missing file: ......\Action_37 change FASTBuild.cs else if (Action.CommandPath.FullName.Contains("link.exe") || Action.CommandPath.FullName.Contains("orbis-clang")) to else if (Action.CommandPath.FullName.Contains("link-filter.exe") || Action.CommandPath.FullName.Contains("orbis-clang"))

but new problem occur 3>1> Exe: ......\Engine\Binaries\Win64\UnrealHeaderTool-BuildSettings.dll 3>EXEC : 1> error : No child command line specified. 3> 3>Usage: link-filter.exe -linker= - - 3> 3>EXEC : 1> 1> Failed to build Exe. error : 1 (0x01) Target: '......\Engine\Binaries\Win64\UnrealHeaderTool-BuildSettings.dll'

I need your help

[Hisamera]

@Peak2018 It has already been mentioned that there are multiple persons developing a working solution, until one lands, you either write one yourself, or wait. Nobody here is gonna help you (or at least it's highly unlikely)

[absolly]

Experimental fastbuild support in unreal engine staging branch: https://github.com/EpicGames/UnrealEngine/tree/release-engine-staging/Engine/Source/Programs/UnrealBuildTool/Executors/Experimental

[ghost]

In UE 4.26 it wont work unfortunately. Tested with Sample Project using two machines.