search

mamba-org / boa

The fast conda package builder, based on mamba
https://boa-build.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
254 stars 56 forks source link

Python based build scripts #216

Open wolfv opened 2 years ago

wolfv commented 2 years ago

I really like the build scripts of the spack project. Instead of writing a bash or cmd.exe script, they have the build instructions in Python. More specifically they have a bunch of classes (CMake, Autotools, Meson, ...) that have certain configuration points. These classes then execute or generate a build script.

Instead of a build.sh we'd have magic handling for a build.py script (that works on Win and Unix).

https://github.com/spack/spack/tree/develop/lib/spack/spack/build_systems

I think taking some inspiration from this approach would be nice, because writing bash or cmd.exe scripts can be quite hard.

wolfv commented 2 years ago

Specifically I am thinking about similar class based approach for common build tools.

Additionally we should have a library with common functionality (cp -r ... to prefix, setting env vars, ...).

The build scripts should be shipped as a seperate package. We might want to develop the basic skeleton in the boa source, but make them customizable via some file so that e.g. conda-forge can add their variables.

Also this would make it simple to support the features better.

bollwyvl commented 2 years ago

I think generally many people would be happier with a real programming language than layers of hacks on config languages. The biggest thing, i think, is to not invent a new DSL for which there exists no tooling.

Probably the final argument of kings in configuration languages would be dhall but it isn't so widely distributed.

But perhaps a "real" syntax doesn't have to mean "the entire ecosystem" or even "batteries included" like CPython, perhaps: rather than a full CPython, perhaps an implementation of a more limited dialect like bazel's starlark would make sense.

There are single-binary go and rust implementations and they offer static and (optional) runtime type checking which kinda sound great for this use case, and have better support for parallelism baked in.

Perhaps building atop this foundation would be more predictable, especially when a job it's going to have is... building CPython!

wolfv commented 2 years ago

For the moment we're experimenting with Python based build scripts in the bitfurnace repo https://github.com/mamba-org/bitfurnace and you can see some ported recipes in this PR: https://github.com/mamba-org/boa-forge/pull/80/files

I think starlark could be interesting as well. I am not quite sure what the benefits are. For the moment with bitfurnace we're actually re-using the Python interpreter that comes with boa so there is no additional heavy python dependency. This would change when we move boa to a pure C++ implementation but that's far out :)