Closed willprice closed 3 years ago
My grand aim is to split the build process into two steps, managed by different makefiles.
Makefile
, will include all the helper targets, packaging etcMakefile
, which will be copied into the _build
folder along with all the necessary source files so that proper dependencies can be set up.Currently proper dependency tracking is (almost) impossible since our current Makefile
downloads and unzips directories as part of the build process. This means it is impossible to use $(wildcard *.ext)
to match files that are produced as part of the download. Technically it is possible to circumvent this issue by hard coding all the files that will be produced as a result of unzipping the files, however this would be rather unpleasant to manage (and would require some Makefile generation machinery to manage in a non-brittle manner).
Instead, I'm proposing (to be completed in a follow up PR) to assemble the skeleton of the _build
directory with all the necessary files (e.g. unzipped packages, base files) which will then enable the writing of a sane Makefile that can make use of wildcard matches to set up deps properly.
One issue I envisage with this approach is in managing the dependencies of base
correctly. Ideally we'd have it so that only the necessary build steps are re-executed when a single file in base
is modified. However, this is made challenging if we copy of base
to _build/poplog_base
in one go. As an alternative we could copy each file individually with proper provenance tracking, e.g.
BASE_FILES:=$(shell cd base && find . -type f -printf '%P\n')
BASE_TARGET:=$(BASE_FILES:%=_build/poplog_base/%)
$(BASE_TARGET): _build/poplog_base/%: base/%
mkdir $(@D)
cp $< $@
However, this is understandably rather slow due to the many thousands of cp
invocations that are spawned.
But without tracking the dependencies on base
, a lot of the benefits of a proper build system are no longer possible. Instead, we'd just end up copying over all of base
again, triggering a rebuild of much of the system... unless the mtime
of the base files are maintained when copying the files into the _build/poplog_base
directory or if rsync
or similar is used to only copy the updated files. This is something I've only just thought of whilst writing this, so I don't know whether it'd work or not.
This PR breaks up the Makefile into a bunch of fragments that are
include
-ed into the mainMakefile
. I'm not 100% sure on this change. On one hand it makes it feel a bit more manageable to split the file up into orthogonal parts. One the other hand, there are dependencies across files, which is perhaps a little counter intuitive.For reference, the semantics of
include
are to(see https://www.gnu.org/software/make/manual/html_node/Include.html for more details on
include
)