Each build takes ~60s, regardless of what changes to source actually occur. This is long enough to be annoying / a serious waste of time, and also to get distracted and take even longer to resume task.
Root cause
The current build system is implemented via a Makefile that does a full project build every time. This is obviously wasteful. Most of the time spent during the build is on the Clang front-end, which doesn't have fine grain metrics. Each compiled source file generates a report like the following (and we have ~60 of them):
This continues to occur even after I depolluted namespaces in #7. So I'd say the root cause here is unnecessary recompilation and parsing.
Approaches
Here are some ideas I have for solving this:
Refactor Makefile
This would involve using Make to implement targets for each individual .o file, compiling them separately, and linking them. The paper Recursive Make Considered Harmful has some good suggestions about how we can do this with include statements so we can have Makefiles in subdirectories but still have fast builds.
Investigate PImpl and precompiled headers
I've seen pointer to implementation proposed as a solution to some situations where slow compilation occurs due to header files, but I don't know if it's relevant here. We also don't currently know the extent to which large system headers are contributing to the parsing slowness. It could be a useful idea to precompile headers like <vector> or <iostream> since they almost never change between Sprawl builds. I should also probably do this with the 3rd party libraries (nlohmann/json is huge).
Switch to a different build system
I need the following from any build system:
Easy to use; I already know how to use Make well enough, and I don't want to have to spend hours debugging build system issues, reading documentation, etc just to get my code to compile.
Minimal indirection between me and the compiler; I want to use Clang, and I want be able to set flags directly.
Ability to compile cross platform easily. This is at odds with the above goal, but I can still build on my Macbook and Linux desktop machine with Make pretty easily. I will want to produce Windows games from Sprawl eventually, but building Sprawl itself on Windows for development is not a priority right now.
Easy to modify toolchain; adding Valgrind and Clang's static analyzer to my Makefile is extremely simple
Easy to add new source files, headers, and their dependencies to the system. I like having lots of small source files and headers since it's easier to think about. Make needs a little boilerplate for this, which I am not against.
Easy to integrate third party libraries. We use libraries which are too large to directly include in the project (SDL and SDL_image) and will use more. I don't mind compiling these into a shared library and providing the headers, but I want integrating these artifacts to be as easy as possible.
Here are some alternatives I've tried/considered, but ultimately rejected in favor of just working with Make:
Bazel
I used Bazel at ATG and in another project. However, the tutorial for modifying the toolchain so I could set Clang flags took ~10 steps, had numerous indirection layers, and a custom DSL. This is way too complex for one person on Mac and Linux.
Meson
I looked at the docs and a couple of other projects using it, but determined it was too complicated as well. It uses a Python-like DSL, and I had to spend a non-trivial time trying to figure out how to specify include directories / header file dependencies for each source file.
CMake
I like CMake and will probably switch to this eventually. However, it has a nontrivial DSL and requires a few tutorials to figure out how to use. Too complicated for now.
Write my own
I briefly considered borrowing by build script from my competitive programming repository and modifying it to check when build artifacts needed to be reproduced, but this boiled down to re-inventing Make.
The Problem
Each build takes ~60s, regardless of what changes to source actually occur. This is long enough to be annoying / a serious waste of time, and also to get distracted and take even longer to resume task.
Root cause
The current build system is implemented via a Makefile that does a full project build every time. This is obviously wasteful. Most of the time spent during the build is on the Clang front-end, which doesn't have fine grain metrics. Each compiled source file generates a report like the following (and we have ~60 of them):
This continues to occur even after I depolluted namespaces in #7. So I'd say the root cause here is unnecessary recompilation and parsing.
Approaches
Here are some ideas I have for solving this:
Refactor Makefile
This would involve using Make to implement targets for each individual
.ofile, compiling them separately, and linking them. The paper Recursive Make Considered Harmful has some good suggestions about how we can do this withincludestatements so we can have Makefiles in subdirectories but still have fast builds.Investigate PImpl and precompiled headers
I've seen
pointer to implementationproposed as a solution to some situations where slow compilation occurs due to header files, but I don't know if it's relevant here. We also don't currently know the extent to which large system headers are contributing to the parsing slowness. It could be a useful idea to precompile headers like<vector>or<iostream>since they almost never change between Sprawl builds. I should also probably do this with the 3rd party libraries (nlohmann/jsonis huge).Switch to a different build system
I need the following from any build system:
Here are some alternatives I've tried/considered, but ultimately rejected in favor of just working with Make:
Bazel
I used Bazel at ATG and in another project. However, the tutorial for modifying the toolchain so I could set Clang flags took ~10 steps, had numerous indirection layers, and a custom DSL. This is way too complex for one person on Mac and Linux.
Meson
I looked at the docs and a couple of other projects using it, but determined it was too complicated as well. It uses a Python-like DSL, and I had to spend a non-trivial time trying to figure out how to specify include directories / header file dependencies for each source file.
CMake
I like CMake and will probably switch to this eventually. However, it has a nontrivial DSL and requires a few tutorials to figure out how to use. Too complicated for now.
Write my own
I briefly considered borrowing by build script from my competitive programming repository and modifying it to check when build artifacts needed to be reproduced, but this boiled down to re-inventing Make.