Make optional use of precompiled headers (PCH) to improve build speed

[dmoody256]

Describe the project you are working on

Build speed can be increased by using PCH files to make complication of a given source file faster. This can also be made transparent and optional through the use of force include options on the compiler command line (/FI or -include). Although thorough analysis can determine a good set of headers to include in a given PCH file, usually just taking the headers that are used most in set of source files will provide satisfactory gains.

Describe the problem or limitation you are having in your project

builds are not as fast as they could be.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

It allows the compilers to compile faster.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

To demonstrate how this will work I will use the editor directory as an example.

You can quickly gather the headers which are most used in the editor directory with a similar script:

import os

includes = {}

for dirpath, dnames, fnames in os.walk("./"):
    for f in fnames:
        if f.endswith(".cpp") or f.endswith(".h"):
            try:
                fin = open(os.path.join(dirpath, f), encoding='utf-8')
                lines = fin.readlines()
            except UnicodeDecodeError:
                fin = open(os.path.join(dirpath, f), encoding="ISO-8859-1")
                lines = fin.readlines()

            if fin:
                for line in lines:
                    line = line.strip()
                    if '#include' in line:
                        if line not in includes:
                            includes[line] = 1
                        else:
                            includes[line] += 1
                fin.close()

for line, num in sorted(includes.items(), key=lambda tup: tup[1]):
    print(f"{line}: {num}")

Taking the top 5~10 headers will provide generally good gains, but depending on the factors, it can be tweaked for better performance. In this case the editor directory yielded:

// editor_pch.h
#pragma once
#if defined(__cplusplus)
#include "scene/gui/dialogs.h"
#include "scene/gui/tree.h"
#include "core/os/keyboard.h"
#include "editor/editor_plugin.h"
#include "editor/editor_scale.h"
#include "editor/editor_node.h"
#endif

Then using a tool to automatically apply force includes in conjunction with SCons PCH option, you can transparent (not needing to modify any existing source) apply the pch:

    editor_env.Tool("forceincludes")

    pchEnv = editor_env.Clone()
    pchEnv['PCHSTOP'] = ''
    pchEnv['FORCEINCLUDES'] = []
    editor_env['PCH'] =  pchEnv.PCH('editor_pch.h')[0]
    editor_env.Append(CPPPATH=['#','#editor'] )
    editor_env['PCHSTOP'] = 'editor_pch.h'
    editor_env.PrependUnique(
        FORCEINCLUDES=[
            'editor_pch.h'
        ],
    )

Note that I had to also propagate the editor_env SConscripts because only the main environment is imported. Generally with SConscripts , its advantages to work within the calling SConscripts environment.

This gave a 20% boost over no PCH for building the editor directory.

If this enhancement will not be used often, can it be worked around with a few lines of script?

It would be desirable to disable the PCH option when doing development on headers in the PCH files. Changes to the PCH headers will cause a rebuild of all source files that use that pch, however since we took the top used headers there is not much difference in things to rebuild when changing some of these headers.

Is there a reason why this should be core and not an add-on in the asset library?

its part of the build

[Calinou]

I suggest adding a SCons use_pch option that defaults to true, so that users can use use_pch=no to disable PCH usage in builds.

[jmgomez]

Any news on this?

[Calinou]

Any news on this?

To my knowledge, nobody is currently working on implementing this. Pull requests are welcome :slightly_smiling_face:

[jmgomez]

@Calinou thanks for getting back to me. Unfortunately, Im not that familiar with Godot and its build system. I was asking for it as part of a POC I wanted to do for binding a lang using GDExtensions.

The lack of PCHs makes the POC fail before even starting it, so I will start by creating it by hand. Do you know if somehow SCon exports all the search paths it gathers or, even better: is there a way to dump all the compiler augments? (So I can build the PCH with matching args)

Edit: -v does it