Tracking issue for feature-parity with nanobind CMake build system

[wjakob]

Dear @nicholasjng,

Thank you for creating a Bazel interface, I think that this will be useful to many people. I took a brief look, and thought to list a few things that are still missing for feature parity:

• It seems to me that the ability to build a shared libnanobind was lost. This is very useful for larger binding projects that partition their extensions across several separately linked binding components.
• In size-optimized release builds on Linux involving a static libnanobind (a very common case), specify -ffunction-sections -fdata-sections to the compiler and --Wl,--gc-sections to the linker. These flags must be specified both when building libnanobind.a, and to the consumer of libnanobind.a to strip away unused parts of the library. On macOS, the linker must be called with -Wl,-dead_strip, no compilation-specific flags are needed.
• In size-optimized release builds, specify -DNB_COMPACT_ASSERTIONS to remove long string constants needed for debugging.
• You need to specify -fno-strict-aliasing to the libnanobind component to avoid miscompilation due to type punning. In contrast to the ones above, this flag does not need to be passed to the consumers of libnanobind.
• It seems that PyPy support was lost.
• Lost the ability of making stable ABI builds.
• It would be good to make the size-optimized build optional (nanobind exposes a NOMINSIZE parameter to do so.). Some uses have reported performance decreases with the default settings because they do serious number crunching directly in the bindings.

[nicholasjng]

Thank you for that list!

I'll try and start with the shared lib and stable ABI support. Though I'm wondering if the latter needs something from rules_python first (or is it really just a #define?).

By the way: Why do I still need to dead_strip the resulting shared lib on macOS even if I use the linker response file?

[wjakob]

Yes, those are for different things.

• The linker response file is needed to support the latest apple linking technology ("chained fixups") in the presence of symbols that are still unresolved when the build system links an extension library. These are Python C API functions that aren't available until a Python process dlopen()s the extension.
• The dead_strip is needed because libnanobind.a contains a whole bunch of stuff that may not be used by a particular project. For example, if your extension doesn't use ndarrays or doesn't use enumerations, then there is no reason to include the library code for those. To my great surprise, that kind of dead code elimination doesn't just happen automatically when you import a static library into a shared library build but needs extra linker flags.

[wjakob]

Stable ABI support requires a #define and a different naming convention for the resulting shared library files (with .abi3.so on linux and an empty suffix on Windows).

[wjakob]

Actually, upon second thought, think that on Windows builds , you might also need to link to a different .lib file. The CMake Development.SABIModule encapsulates this platform-dependent logic.

[nicholasjng]

Good news - I managed to easily build and run the nanobind_example using my local nanobind_bazel repo. I pushed the necessary changes just now, and will update documentation and set up CI shortly.

What's interesting is that in a setuptools build (rules_python does not support E2E wheel buils with C/C++ extensions yet if I understand correctly), current_python_cc_headers grabs the correct headers regardless of which toolchain is configured in the MODULE.bazel, so I get the right headers and libs for multi-Python builds like cibuildwheel out of the box.

[nicholasjng]

Size-optimized builds are now controllable by a feature flag, see #3.

[nicholasjng]

SABI builds for CPython>=3.12 were added in #5.

[nicholasjng]

I think all points (outside of PyPy support) are addressed now. I'm going to leave this pinned on the Issue tab and create a followup, but this seems pretty much complete.