Add a document explaining nanobind extension building using Bazel

[nicholasjng]

This is designed to be a better-exposed (re: SEO) doc on how to use nanobind with Bazel, aimed at people who would like to get started with nanobind but do not want to search around reading outside documentation.

Contains sections on declaring the dependency, importing nanobind-bazel's APIs, how to use it in BUILD files, and lastly how to build extensions with Bazel.

Mentions a few noteworthy special points of information, such as constraints on the Bazel version, overriding versions with bzlmod APIs, etc.

---

This is a first draft, I still need to finish the sections on the APIs and the extension builds themselves.

Happy for comments - especially regarding depth, style, and flow. Ideally (in my opinion), the user should be able to get started by creating (at minimum) a MODULE.bazel and a BUILD file in their project with zero outside clicks.

Also still missing: Python packaging. I don't have a good example of a wheel build with Bazel with C++ extensions in the mix, I could present the setuptools config from the nanobind example, though.

[nicholasjng]

Also, let me know your preferred grouping of the new document - I first thought to place it in the last group, but it's not really just an API reference - it doesn't really fit into any other category either, though. I'm still leaning toward the API reference group.

[wjakob]

Hi Nicholas, this looks great. Compared to the CMake documentation, what would be nice to add are details on parameters to customize the build (e.g. shared/static, stable ABI, etc). One option could also be to split up the documentation into a tutorial and a reference part -- that way, the information is all there and it stays digestible -- I leave it up to you. +1 for saying at least a bit about packaging. The rest of the nanobind documentation is focused on the scikit-build-core workflow that I assume does not work with bazel.

[henryiii]

Sideline question: is there something like scikit-build-core (CMake) or meson-python (Meson) or maturin (Rust) using Bazel, allowing wheels and SDists (and therefore installing, editable installs, etc)? I've not seen one and I (simi-)regularly inspect statistics on every build backend specified in an SDist on PyPI. Oh:

Also still missing: Python packaging. I don't have a good example of a wheel build with Bazel with C++ extensions in the mix, I could present the setuptools config from the nanobind example, though.

I take it the answer is likely no. If someone wanted to work on one, there will soon be a wheel helper to packaging, which might simplify writing a new one.

Without a custom tool, by far the easiest way to do this is with hatchling, which has a designed API for writing custom builds like this. It doesn't have nice a nice utility for Stable ABI wheels yet, but I'm planning on contributing that eventually (and it's easier than setuptools even if you manually customize the tag!).

[nicholasjng]

Sideline question: is there something like scikit-build-core (CMake) or meson-python (Meson) or maturin (Rust) using Bazel, allowing wheels and SDists (and therefore installing, editable installs, etc)? I've not seen one and I (simi-)regularly inspect statistics on every build backend specified in an SDist on PyPI. Oh: [...] I take it the answer is likely no. If someone wanted to work on one, there will soon be a wheel helper to packaging, which might simplify writing a new one.

Correct, at least to my knowledge. I dove into scikit-build one time to see if it was feasible starting to write one myself, but I did not go through with it for the time being. Aside from that, I think this is by design: Bazel expects to be in charge of all parts of the build (i.e. extensions as well as Python files, data, stubs, etc.), so the "canonical" way to build a wheel would be to run bazel build on a wheel target.

An example of a wheel rule can be found here: https://github.com/bazelbuild/rules_python/blob/b07525cbb15352caefbe2f23697250cecd984430/python/private/py_wheel.bzl#L504-L510

(You'll notice that you can specify a _wheelmaker, which is basically the Python script used to build the wheel. Some projects roll their own, though, for example JAX.)

Without a custom tool, by far the easiest way to do this is with hatchling, which has a designed API for writing custom builds like this. It doesn't have nice a nice utility for Stable ABI wheels yet, but I'm planning on contributing that eventually (and it's easier than setuptools even if you manually customize the tag!).

Nice - even with the above in mind, I think it's still a very good idea to have a facility to build wheels tied to a Python interpreter. Bazel's hermeticity guarantees are critical to its mission, but I think that relaxing on it a bit for a Python package build might be a good idea. Not to even begin talking of software support - if I remember correctly, it's still impossible to produce statically linked Python extensions with rules_python because of bugs related to the LD_PATH of the hermetic Python distribution libs it uses.

Would you happen to have a tip on how one would start with a "scikit-build for Bazel"?

[henryiii]

Bazel expects to be in charge of all parts of the build

That makes it easier, actually - it's more similar to meson-python, and much more similar to maturin - maturin implemented everything, including wheel and sdist generation, in Rust, and just has a simple Python wrapper to adapt PEP 517. You'd just need to implement a few simple Python hooks (see PEP 517) which would call Bazel; it's just a function to make wheels and a function to make SDists at minimum. SDists might be a little harder, but I've implemented them the same way hatchling implements them; include everything that's not in a .gitignore file by default, and include the .gitignore file so SDists can make SDists. You also want metadata, but if Bazel can fill all the possible slots, then you don't need PEP 621 or pyproject-metadata unless you want it (or can teach Bazel to read it). Most of the complexity in scikit-build-core (which isn't that complex) comes from the configuration system, as it's quite flexible. I also spent quite a bit of code on a FileAPI reader that we still haven't used anywhere.

We have monthly scikit-build community meetings (bi-monthly if you count the developer meetings too, which are also public), if you'd like to join one I'd be happy to talk about what would be required. Link and info in the scikit-build-core docs and readme.

[nicholasjng]

We have monthly scikit-build community meetings (bi-monthly if you count the developer meetings too, which are also public), if you'd like to join one I'd be happy to talk about what would be required. Link and info in the scikit-build-core docs and readme.

Thanks, will do. Next one should be Friday the 21st, right? I'll read up on PEP517 and scikit-build-core again until then.

[henryiii]

I might actually miss that one, I'll be off and have family over, and exactly at that time I've got to pick up someone from the airport. If I'm sitting in the cell phone lot, might dial in. One of the Kitware people might miss it too. I'd recommend either of the meeting next month (though you are welcome to drop by the one next week and say hi to whoever is there!)

[henryiii]

Looks like I will be able to be at tomorrow’s meeting. Feel free to join if you can.

[nicholasjng]

Hi Wenzel, I just finished the first draft of the building + API reference split setup that you suggested. I'd be happy for a review when you find the time!