[Feature Request] Automatically generate Python bindings from Mojo package

[jordiae]

Review Mojo's priorities

• [X] I have read the roadmap and priorities and I believe this request falls within the priorities.

What is your request?

When running mojo package mypackage, Mojo should automatically generate Python bindings, in the form of an automatically generated __init__.py and mypackage.whl even.

Unlike other bindings for Python such as PyO3 [1] in the case of Rust, given Mojo's syntactic and semantic compatibility with Python, my uninformed guess would be that it should be possible not only to offer a way to develop Python bindings, but to generate them automatically given a Mojo API specification defined in __init__.mojo.

[1] https://github.com/PyO3/pyo3

What is your motivation for this change?

Enable easy Python -> Mojo interoperability (rather than the already supported Mojo -> Python).

Any other details?

I guess something along the lines is already in Modular's roadmap, but haven't seen it mentioned in the official website or documentation.

[gdonval]

Maybe not automatic (I don't think it would be a sane default) but a flag/verb away could be neat. (I say this as someone who would absolutely love to use Mojo modules from Python, don't get me wrong!)

Just so you know, the problem is a bit tricky on that side: you have to make a module that is understandable by cpython (basically generating a C-compatible ABI hooking into cpython, which is not trivial). It's not just a matter of doing the boxing/unboxing dance the other way around. I really don't mean to start a discussion on this: that way around is not trivial even if you can already do the other way around, I wanted to make that clear. That's all.

Folks working on anything bigger than a script will also want a way to provide fine-grained public APIs. It's relatively easy to provide meaningful semantic versioning with a clear and limited API for instance but if you expose everything, all bets are off (and your binary will bloat in no time).

There are solutions to these, I know... and Modular folks know too! This is not a blocker, this is not a way to say it won't be implemented (I am sure it will when mojo's sources are released, at the latest!), I am just saying you should temper your expectations in the short/medium term because this is not as high-priority as traits for instance (which are required for the language to be complete) and it's not as trivial as it might appear at a first glance.

[jordiae]

Maybe not automatic (I don't think it would be a sane default) but a flag/verb away could be neat. (I say this as someone who would absolutely love to use Mojo modules from Python, don't get me wrong!)

Just so you know, the problem is a bit tricky on that side: you have to make a module that is understandable by cpython (basically generating a C-compatible ABI hooking into cpython, which is not trivial). It's not just a matter of doing the boxing/unboxing dance the other way around. I really don't mean to start a discussion on this: that way around is not trivial even if you can already do the other way around, I wanted to make that clear. That's all.

Folks working on anything bigger than a script will also want a way to provide fine-grained public APIs. It's relatively easy to provide meaningful semantic versioning with a clear and limited API for instance but if you expose everything, all bets are off (and your binary will bloat in no time).

There are solutions to these, I know... and Modular folks know too! This is not a blocker, this is not a way to say it won't be implemented (I am sure it will when mojo's sources are released, at the latest!), I am just saying you should temper your expectations in the short/medium term because this is not as high-priority as traits for instance (which are required for the language to be complete) and it's not as trivial as it might appear at a first glance.

Thanks for your answer!

Sure, I didn't intend to imply that it would be easy at all for Modular, but for the end user.

The developer should indeed be able to have a way to manually define the Python API, and manual bindings would still be needed for some Mojo constructs due to the complexity or impossibility of directly exposing them to Python, but still, my uneducated guess would be that automatic binding generation should be possible for at least some Mojo constructs. While potentially very complicated to implement for Modular, this would greatly enhance Mojo's adoption.

[jd-solanki]

I'm eager to build libs using mojo like we have more performant libs rewritten in rust.

Can't wait for mojo to provide python bindings.