Add new build system

[VincentVanlaer]

Main commit message

Reasons for replacing the previous build system were:

• the previous build system had a lot of duplication between the modules, both in terms of make files and shell scripts.
• parallelisation was limited. The new build system parallelises better within a module, and between modules.
• build scripts and output from the build ended up in the same folder. With the new system, all build output files are stored in the folder build in the repository root, making cleaning build files equivalent to deleting this folder

For the details of the implementation, see the newly added documentation (Developing -> Build system).

Some further notable changes:

• makedepf90 has been replaced by a perl script that scans the fortran source files. This allows for better flexibility in construction the make files.
• Currently, pymesa does not work with the new build system, as the new system does not provide a way to build shared libraries for every module.
• Dependencies are being discovered using pkg-config. This change makes it easier to build MESA without the SDK, as linux distributions typically ship with pkg-config files.

Draft status

Draft status of this PR will be removed once a decision has been mode about #712 and #723, after which I will rebase.

Reviewing this PR

I recommend to first take a test drive of the new system, especially if you use some special setup (e.g. a non-sdk build). Install instructions haven't changed. I have written relatively detailed documentation on how the new system works, which I recommend reviewers of this PR to read before diving into the Makefiles and scripts. Let me know if something is unclear or missing in the documentation! The first few commits are just #712 and #723, so those can be skipped (if #712 and #723 don't end up being merged, it is an easy change to add the files back). The other commits are truly part of this PR, with the first two introducing the new build system. The other commits are porting of individual modules.

Remarks

• About half of the modules have been ported.
• There is a lot of output on each run for installed files to $MESA_DIR/lib and $MESA_DIR/include. Once all modules have been ported, this will no longer be needed for a normal install, except for the data files. Internally, the different modules that have been ported don't use those files.
• I generate version numbers using git describe --tags which generates (in my opinion) nicer version numbers than just using the commit hash. However, this requires the version tags to be on the main branch (currently running this command on main gives r15140-2084-ge29ed74d rather than something like r24.08.1-NNNN-ge29ed74d).
• It would be beneficial to add GitHub actions for the following cases
  • Building without SDK
  • Building all helper programs which are not built by default
• As mentioned in the main commit message, pymesa currently doesn't work. This is because I think it would make more sense to build one single shared library after the main build as a separate step, rather than a bunch of little libraries which need to be able to refer to each other. I intend to do this anyway for the output of the star/binary module, such that building the actually star binary can proceed faster, and allow for things such as experimenting with link-time optimization.

Future iterations

I have thought about some future iterations of the build system after this, but I have not implemented those things since otherwise I would keep iterating forever on this build system trying to design the perfect build system. Hence, writing this down to at least have it documented somewhere.

1. Use a single make invocation to build and test all modules. Currently, each module is built in a separate make invocation, which limits parallelisation and true dependencies between modules (once the mod files have been generated, the dependent MESA modules could start building). Furthermore, if you change something in an include file which is included in another module, make might not immediately pick it up. Combining all modules into one make invocation would prevent such surprises in the development. In order to not cause excessive rebuilds, it would be necessary to use Fortran submodules, such that no public modules are dependent on their implementation. All of this requires a full set of generated make files, as to not overlap the variables and targets used by the different modules.
2. With such a system in place, it may be beneficial to replace the generated make files by ninja. The main issue with the system from 1. (and in fact the current system as well) this tries to solve, is that if you remove an include file, make will error out. This is because the include file is in the dependencies of the module. From what I understand, by using dynamic dependencies in ninja, it will not error out in such a case. However, this very much untested. In general, I think it is possible to make a build system in this way that exactly compiles what is needed, and doesn't error if cleaning and building wouldn't cause an error as well. Maybe there are easier ways to fix that in the end than using a different command runner.

I have probably forgotten some things I wanted to mention, but this should be most of it.

[warrickball]

I've just started chipping away at this and started off with the error

mkdir -p build
make/gen-folder-deps "make" const utils math mtx auto_diff num interp_1d interp_2d chem eos forum > build/depend
fatal: No names found, cannot describe anything.
make: *** [make/version.mk:2: data/version_number] Error 128
make: *** Deleting file 'data/version_number'

for both make and ./install, which emanates from git tag. I couldn't for the life of me fetch any tags so I just tagged the HEAD commit on your build-system branch and that got the build going.

I'll see how far I get with the SDK (the build feels faster) and then give it a try with my system libraries, which includes GCC 14.2.1, and then the ifort build on the local cluster. It'll take me at least 2-3 weeks to work through but is my MESA priority.