Provide support for HPy

[thedrow]

HPy is a better Python API written in C. The project's goal is to hide CPython's implementation details so that any Python implementation will be able to support C extensions with ease.

One of the main goals of the project is to allow PyPy to support C extensions without emulating CPython's API.

We should consider providing a backend that uses HPy instead of directly using the CPython API. We can use a feature toggle to activate it.

Since HPy's API can be mixed with CPython's API we can gradually migrate more and more code to HPy's API as it evolves.

A first attempt to provide Rust bindings to HPy was made in pyhandle/hpy#20.

[davidhewitt]

It would be interesting to see support for it. One other consideration is that CPython is aiming to move itself towards a much leaner API which might be easily supported by PyPy (https://pythoncapi.readthedocs.io/)

[thedrow]

The authors of that document are part of the HPy effort.

[adsharma]

This is of interest to py2many, a python -> rust transpiler (apart from 6 other languages).

https://github.com/adsharma/py2many/issues/62

[cirospaciari]

Any plans or roadmap to supporting HPy? CPython API slows down a lot in PyPy

[davidhewitt]

While this would be cool, it's low on the list of things which I would to be getting to, so help needed realistically for the foreseeable future.

[cirospaciari]

While this would be cool, it's low on the list of things which I would to be getting to, so help needed realistically for the foreseeable future.

I see, thanks for the update, Thinking of starting HPy.rs project to be a 1-to-1 with HPy., I will put it in my work queue.

[thedrow]

@cirospaciari The HPy project will likely accept a contribution that provides a Rust API automatically using code generation.

[davidhewitt]

That sounds like a great first step, if the HPy project generated an hpy-sys crate (perhaps) which could be used instead of pyo3-ffi when desired.

[cirospaciari]

That sounds like a great first step, if the HPy project generated an hpy-sys crate (perhaps) which could be used instead of pyo3-ffi when desired.

I will try to do this ;)

[messense]

I've put together a rough hpy-sys crate generated via bindgen: https://github.com/messense/hpy/tree/rust/hpy/devel/rust/hpy-sys, there are some issues with this approach, mostly related to cross compilation:

• Hard dependency on libclang
• Hard dependency on a working Python interpreter when using cpython or hybrid abi modes

I'm thinking about generating the ffi bindings using bindgen cli for each abi modes instead and commit them in git to avoid these issues.

[mattip]

@cirospaciari how is the work to support HPy going? Is there anything I could do to help?

[mattip]

@messense in hpyproject/hpy#325, @thedrow shows some bindgen output. I don't know rust, is this a step in the right direction?