waruqi
commented
1 month ago Now we see some of the less-nice bits of xmake. You can't extend the xmake declarative syntax, you can only add properties in the imperative world, so we now need to write a Lua function that runs when this target is loaded.
Xmake supports custom description scope interfaces. You can customize the firmware and compartment scope api, as well as all configuration interfaces, like this:
https://github.com/xmake-io/xmake/blob/dev/xmake/includes/xpack/xmake.lua
https://github.com/xmake-io/xmake/issues/1433#issuecomment-1811761871
Targets in xmake must have globally unique names.
v3.0 will provide namespace to support it. https://github.com/xmake-io/xmake/issues/5527
If you add a compiler flag, xmake will drop it if it isn't supported. It doesn't try passing it to the compiler to see if it's supported
you can disable this policy. set_policy("check.auto_ignore_flags", false)
davidchisnall
sleffler
boris-kolpackov
We originally built CHERIoT RTOS with CMake. This had a number of problems because CMake is not designed to allow users to build abstractions. We replaced it with xmake prior to open sourcing, and xmake is mostly fine.
Requirements
We need to be able to build four kinds of things:
Ideally, we also want to be able to add other composable chunks, which can be made dependencies of compartments or provided separately, including:
Shared libraries and compartments have a custom link step, as do firmware images.
Firmware images need to be able to walk their dependency graph and build a linker script containing all of the libraries and compartments.
Board description files (JSON, kind-of JSON5, could be YAML) need to be parsed to generate compiler flags, inputs to the firmware linker script, and so on.
The build system must be able to generate
compile_commands.jsonfiles for IDE integration. It's 2024, this is no longer a 'nice to have' optional extra.For secure build environments, we must be able to do builds from a read-only mount of the source tree. In-tree builds are an antipattern.
We should be able to build abstractions that allow very concise definitions of projects. The current xmake build system is fairly good here. Going through the simplest hello-world example line by line:
This is slightly redundant because we need to name firmware images and so the overall project name is not very meaningful.
This is quite ugly. We are forced to hard-code the path of the SDK (more on this later) and then, because of xmake's scoping, also specify the toolchain here, even though the SDK files already specified it.
This is necessary because we don't advertise the board option from the SDK, we make each firmware select it. This is probably the right choice. Some projects may want to build firmware for different projects (the build2 proposed solution here is to have different build configs for each), others may not allow the board to be an option because they're specialised for a single SoC / board.
Again, this is concise. We are defining a compartment called
hello, it depends on thefreestandinganddebuglibraries (could be compartments). It has a single source file,hello.cc. This will be compiled with the default options for a compartment and linked as a compartment.We're starting to define a firmware image called
hello_world, it depends on thehellocompartment.Now we see some of the less-nice bits of xmake. You can't extend the xmake declarative syntax, you can only add properties in the imperative world, so we now need to write a Lua function that runs when this target is loaded.
In declarative syntax, this would be simple
set_board("$(board)"), but that's a small issue.This defines a single thread, its entry point, and so on. It would be nice to have this be a type that could be automatically type checked (the build2 prototype does this), but this is largely fine.
The really ugly thing here is the
{ expand = false }. This is not properly documented in the xmake docs, but is essential to avoid xmake flattening the object that you've put here.Problems with xmake
Although xmake works, we have encountered a lot of problems over time.
Poor defaults
A build system should loudly complain if things are wrong.
If you add a nonexistent file (for example, add it with a
.cextension instead of.cc, or fail to commit it to git), xmake does not complain. It simply builds and, if the link step works, the build succeeded. You probably didn't need the file anyway. We had a test that the FreeRTOS compat headers all compiled in C mode that was spuriously passing for months as a result of this (I added the xmake bits, forgot to add the .c file, and CI passed).By default, xmake hid compiler warnings (this was fixed in v2.8.7). This is the worst possible default. The argument from the maintainer was that 'most users' didn't need to see them. I believe that most users of a build system are developers and they absolutely need to see warnings. The second most common users are people building packages, and they should see warnings because they may help debug problems. The third category are people who are building things from source but are not developers, and these people are a rounding error and probably don't care either way.
If you add a compiler flag, xmake will drop it if it isn't supported. It doesn't try passing it to the compiler to see if it's supported, it has some internal model of the compiler (which may or may not be relevant for a cross-compile toolchain) and ignores it. You must add
{force = yes }(which was undocumented the first time I needed it, I'm not sure if it's documented now) to make xmake do the thing that you explicitly told it to do.Similarly, if you pass a positional argument with any of the flag-adding functions, even with
{force = true}, and xmake doesn’t know about the argument, then it will deduplicate it. Things like-Xclangor-mllvm, which need to be passed multiple times are silently removed. There is no documented way of avoiding that, but there is an undocumented way, which is to pass the pair of flags as a Lua array and addexpand = falsein the last argument.Everything in xmake is a string. The type of a build rule is a string. If you make a typo, it will silently make the target use the default rule.
Insecure searching
Unless you explicitly specify a target path, xmake looks up the directory tree to find the
xmake.luafurthest from you. This may be across a mount point and controlled by a different user. This is an incredibly insecure default.Poor scoping
Targets in xmake must have globally unique names. Some things are local to the file, others are not. For example, you must set the toolchain in the top-level file, you can't provide it in the SDK file.
Extensions are second class
Internally, xmake defines a lot of different target types. These have access to a load of infrastructure that is exposed. Anything that is implemented externally lacks access to this infrastructure and so cannot work as well as things built into the build system.
Opaque dependency tracking and poorly specified execution order
We have several custom rules. It's not clear how to tell xmake that a given file is a dependency of them. Sometimes, the only way to get a safe rebuild is
rm -rf .xmake build.xmake also provides its own caching, which is not invalidated when the toolchain is updated. This breaks reproducible builds and introduces bugs when a new toolchain feature is required.
Options are not processed until after the declarative syntax is processed (and the declarative syntax isn't really declarative). This means that you can't use an option to provide the SDK path. We'd like to be able to have build files that have a default path for the SDK, but let you override it. With xmake, we need to modify
xmake.lua.Similarly, it's unclear what order the
on_loadfunctions for different targets will run in. We tried using the support that was added for dynamically cloning targets for the allocator, but this was impossible because the new target'son_loadthings didn't run.There probably is some internal model for how builds work, but the author has not written it down anywhere.
Other options
There are a few potentially promising alternatives that we should explore before we do a 1.0 release and need to support whatever we ship for a long time. For some reason, they seem to be things that match the pattern
B.*2.Build2
Build2 is a fairly new build system that is designed to support modern C++ features. @boris-kolpackov has built us a proof of concept that can build the hello-world example. We need to:
compile_commands.json(not yet working out of the box and a must-have feature).It looks quite promising. I don't like the fact that compilation is conceptually a separate step. To me, compilation flags depend on the target (library, executable, whatever) and the fact that you have to declare separate
cxxthings is not ideal.I also don't like the focus on short names in build2. These are hard to search for and they place a higher cognitive load on the programmer. Most developers will spend 1% of their time touching the build system, so a build system needs to be easy for people to context switch back to. I find a lot of the build2 logic hard to read.
I also don't like the fact that every project needs three files in two directories to build.
Buck2
Buck2 is a rewrite of Facebook's Buck build tool. It uses Starlark, a Python-like language, for scripting and exposes all of the rules, which means that our build rules would be equivalent to first-class rules from other places.
Buck2 is written in Rust and provides a single statically linked binary that is easy to deploy.
We currently have no proof-of-concept in Buck2, but it is probably worth exploring.
Others?
CMake was used originally and its limitations were too great.
Bazel has some nice properties, but it has too big a dependncy chain. Bazel requires both a working Java VM and a Python installation. This is fragile and far too big a supply-chain attack surface for something in every build.
tup / make are low-level tools that rely on third-party things if you need abstractions. We don't want to maintain a tup generator.
Any others that we should evaluate?