Link files are also response files!

[fire]

Apparently link commands are also response files.

CorePrivatePCH.h.pch.response

3>  5> DLL: E:\Nexa\UnrealEngine\Engine\Binaries\Win64\ShaderCompileWorker-Core.dll
3>5> LINK : fatal error LNK1181: cannot open input file 'E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\Core\CorePrivatePCH.h.obj'
3>

[fire]

"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\Core\CorePrivatePCH.h.obj" doesn't get compiled

[fire]

Maybe I'm seeing it wrong, here's a log.

https://gist.github.com/fire/0f6323d6459498773ff9ae6cfd02c231

[liamkf]

Hmm, that looks the same as the previous 4.13 problem in that it's not getting the command line properly to compile the files, I should have fixed with this change.

If you take the latest and then rebuild UnrealBuildTool and then the ShaderCompileWorker, what do you get?

[fire]

If it helps here's my patches to 4.13.

0002-Load-response-file-to-support-UE4.13.patch.txt 0001-Add-fastbuild.patch.txt

[fire]

git diff --ignore-all-space
diff --git a/Engine/Source/Programs/UnrealBuildTool/System/FASTBuild.cs b/Engine/Source/Programs/UnrealBuildTool/System/FASTBuild.cs
index 84d25bb..1a5b926 100644
--- a/Engine/Source/Programs/UnrealBuildTool/System/FASTBuild.cs
+++ b/Engine/Source/Programs/UnrealBuildTool/System/FASTBuild.cs
@@ -123,10 +123,8 @@ namespace UnrealBuildTool
                 try
                 {
                     if (File.Exists(responseFilePath))
-                    {
                         RawTokens = File.ReadAllText(responseFilePath).Split(' '); //Certainly not ideal
                 }
-                }
                 catch (Exception e)
                 {
                     Console.WriteLine("Looks like a reponse file in: " + CompilerCommandLine + ", but we could not load it! " + e.Message);
@@ -454,11 +452,6 @@ namespace UnrealBuildTool

             AddText("Settings \n{\n");

-            if(envVars.ContainsKey("FASTBUILD_CACHE"))
-            {
-                CachePath = envVars["FASTBUILD_CACHE"];
-            }
-
             // Optional cachePath user setting
             if (CachePath != "")
             {

My changes to FASTBuild.cs.

By the way reponse is spelt response.

I don't see much difference...

[liamkf]

Yes that seems like it should work, but it just occurred to me that there might be spaces in your path! Which would not work... is that the case? I'll come up with something more robust to detect the "response file only" case.

If that's not the issue though... I think setting a breakpoint on the RawTokens.Length == 1 and examining why it's not loading the response file will be the best bet.

[fire]

        private static Dictionary<string, string> ParseCommandLineOptions(string CompilerCommandLine, string[] specialOptions)
        {
            Dictionary<string, string> ParsedCompilerOptions = new Dictionary<string,string>();

            // Some tricky defines /DTROUBLE=\"\\\" abc  123\\\"\" aren't handled properly by either Unreal or Fastbuild, but we do our best.
            char[] SpaceChar = { ' ' };
            string[] RawTokens = CompilerCommandLine.Split(' ');
            List<string> ProcessedTokens = new List<string>();
            bool QuotesOpened = false;
            string PartialToken = "";
                        Console.WriteLine("rawtokens" + String.Join(", ", RawTokens)); // CHANGED HERE 

        private static void AddCompileAction(Action Action, int ActionIndex, List<int> DependencyIndices)
        {
            string CompilerName = GetCompilerName();
            if (Action.CommandPath.Contains("rc.exe"))
            {
                CompilerName = "UE4ResourceCompiler";
            }

            string[] SpecialCompilerOptions = { "/Fo", "/fo", "/Yc", "/Yu", "/Fp", "-o" };
            var ParsedCompilerOptions = ParseCommandLineOptions(Action.CommandArguments, SpecialCompilerOptions);
            Console.WriteLine(string.Join(";", ParsedCompilerOptions)); // Changed here

gives

2>------ Build started: Project: ShaderCompileWorker, Configuration: Development_Program x64 ------
2>  Creating makefile for ShaderCompileWorker (UnrealBuildTool.exe is newer)
2>  rawtokens, @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\SharedPCHs\Core.h.pch.response"
2>  [InputFile, @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\SharedPCHs\Core.h.pch.response"];[OtherOptions,  ]
2>  We failed to find /fo, which may be a problem.
2>  Action.CommandArguments:  @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\SharedPCHs\Core.h.pch.response"
2>  We have no OutputObjectFileName. Bailing.
2>  rawtokens, @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\Core\CorePrivatePCH.h.pch.response"
2>  [InputFile, @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\Core\CorePrivatePCH.h.pch.response"];[OtherOptions,  ]
2>  We failed to find /fo, which may be a problem.
2>  Action.CommandArguments:  @"E:\Nexa\UnrealEngine\Engine\Intermediate\Build\Win64\ShaderCompileWorker\Development\Core\CorePrivatePCH.h.pch.response"
2>  We have no OutputObjectFileName. Bailing.

[liamkf]

Ohhhhh now I see, I had a change sitting staged that apparently did not make it. Just pushed a new version, please let me know if it works!

There's a leading space with the response file for whatever reason... :/ sorry about that!

Also thanks for reporting it! I'll make a note that I'll need to support spaces in paths as well, even though that wasn't your issue.

[fire]

I expect it to work now. I completed shaderworker and currently testing compiling the entire engine.

I think it's a good addition to have the fastbuild's cache location be defined in the environmental settings rather than editing the c# source.

Full set of patches here.

0001-Add-fastbuild.patch.txt 0002-Load-response-file-to-support-UE4.13.patch.txt 0003-Add-the-wayward-.Trim-fix-typo.patch.txt