search

vesoft-inc / nebula-python

Client API of Nebula Graph in Python
194 stars 76 forks source link

Consider migrating to Rust based Python package? #319

Open haoxins opened 7 months ago

haoxins commented 7 months ago

General Question

More and more Python packages are built from the Rust code base. Such as:

  1. https://github.com/pola-rs/polars
  2. https://github.com/Qiskit/rustworkx
  3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages. This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula. The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase. This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are: https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

haoxins commented 7 months ago

Another example from Temporal SDK core

QingZ11 commented 7 months ago

It's really a great idea, thank you Haoxin.

wey-gu commented 7 months ago

General Question

More and more Python packages are built from the Rust code base. Such as:

  1. https://github.com/pola-rs/polars

  2. https://github.com/Qiskit/rustworkx

  3. https://github.com/pydantic/pydantic-core

The associated tools are:

https://github.com/PyO3

The benefit is that we can build the logic from Rust once and publish it as Rust, Python, and even JavaScript (Node.js) packages.

This is especially the trending solution for the data/ML ecosystem.

When I start to build the services and SDKs based on the Nebula.

The first candidate language is Rust so that I can develop and deploy the services as Rust code, and build the Python SDKs (for data scientists) based on the Rust codebase.

This is the ideal solution for writing code only once.

But the Rust client for Nebula is not ready at last year.

Since we already have the first draft version of the Nebula Rust client, I think this is a good time to share this idea with you guys.

The other tools for more languages that I'm not used to are:

https://github.com/dtolnay/cxx for c++

cc @Nicole00 @wey-gu

Agreed, was considering pyo3 when facing perf issues in https://github.com/wey-gu/nebula-dgl/issues/10

We could do so in free time!

haoxins commented 7 months ago

Also find this https://github.com/jni-rs/jni-rs for Java :)

wey-gu commented 6 months ago

Also, recently I had some chance to implement/build something with cython, later I could prepare a PR to add cython build of nebula-python and come with a benchmark.

Abeautifulsnow commented 3 weeks ago

👍Both Rust and Cython are excellent choices, but Rust (using PyO3) may be a more popular and convenient option when integrating with both Python and Node, especially if there already has a mature Rust-based repository here. Hope to see it soon.

wey-gu commented 3 weeks ago

https://github.com/nebula-contrib/rust-nebula

cc @BeautyyuYanli @PsiACE

PsiACE commented 3 weeks ago

https://github.com/nebula-contrib/rust-nebula

cc @BeautyyuYanli @PsiACE

Cool idea. We can start by simply checking that the core functionality is aligned.