[core] Using pybind11 to replace Cython bindings

[hongchaodeng]

Description

Problem

Currently we use Cython as the glue layer between Python and C++ (core worker) code. This has several problems.

Cython was better suited for creating simple wrapper around C code. But in current architecture, the code is complex and some use cases are beyond Cython design.

Here are some pain points:

• We are not only calling C++ code from Python, but also calling Python code from C++. To do that, we have to use constructor to setup callback pointer to work around that.
• There are two compiler phases: Cython compiled to C code, then C code compiled again. Some bugs will only get caught in the second compilation pass, after Cython has generated thousands of lines of hard-to-decipher code.
• Cython code is unsafe. It can lead to the same memory issues (e.g. segfault) as C++ without the modern C++ safety features.
• Cython code maintenance cost is high. It is lack of tooling and IDE support. Lots of glue code in build system.

Proposal

Proposing to use pybind11 to replace Cython bindings. It has the following benefits:

• It integrates seamlessly with modern C++
• It is testable in C++
• It supports arbitrary exception types and good translation of exceptions between C++ and Python
• Integrated NumPy support

We can do this incrementally.

Use case

No response

[Superskyyy]

The Pybind11 overhead is not neglegible in Ray, how about considering https://github.com/wjakob/nanobind that comes from the author of Pybind11 and claim to deliver 10x less overhead? That way we get benefit from an almost identical syntax with minor overhead. @rynewang @hongchaodeng

Some background https://nanobind.readthedocs.io/en/latest/why.html#why-another-binding-library

[rynewang]

cc @dentiny

[rynewang]

From a quick glance, this nanobind is similar to pybind11 in architecture, just with some optimizations?

[Superskyyy]

From a quick glance, this nanobind is similar to pybind11 in architecture, just with some optimizations?

Exactly, it's the same author, nanobind dropped some historical technical debts so being more performant. And the author suggests using Nanobind instead of Pybind11 unless absolutely needed.

[dentiny]

Can I understand as, the biggest gain is to leverage modern C++, since cython leverages C while pybind for C++?

Some bugs will only get caught in the second compilation pass, after Cython has generated thousands of lines of hard-to-decipher code.

For this point, I'm curious how pybind11 helps here? When wrap C++ code via pybind11, we could rely on compilation in one iteration to check; but it's hard to decipher as well, since pybind heavily rely on templates; When wrap (or say, load) python objects via pybind, everything should be at runtime.

I have some bad experience and memory for python/C++ FFI overall, for example,

• Tricky to decide whether safe to release GIL
  • GIL is used to protect cpython internal data structures, so in theory all accesses to python internal type should have GIL held, not always easy to write and justify
• Overall hard to debug and profile for both python and C++
  • For example, gdb and coredump show cpython internal call stack, same as flame graph

Just curious, do you think it's possible to leverage localhost network instead of FFI? I definitely understand network call is one order of magnitude slower than FFI, since it involves more copies, but

• pybind requires copy as well, for example, when you want to pass a string from C++ to python, likely you have to rely on pybind11::bytearray-like data structure, which takes a copy: https://github.com/pybind/pybind11/blob/f46f5be4fa4d24c4e5382d0251315f361ce97424/include/pybind11/pytypes.h#L1766-L1791
• (correct me if wrong) We use cython to transmit infrequent lightweight control plane message; if we use localhost network, the latency would be ~100 microsec magnitude, which is cheap compared with our 100 millisec task execution time.