Open gerrymanoim opened 5 months ago
👍 That definitely seems like the way to go for performance and flexibility. Rust is still stuck on my todo list ☹️
I'm wondering to what degree it would complicate the build and installation?
I'm wondering to what degree it would complicate the build and installation?
pyo3/maturin (https://www.maturin.rs/) are pretty well supported these days. We'd basically build wheels for everything and most users would just install the wheel (https://github.com/pydantic/pydantic-core/blob/main/.github/workflows/ci.yml#L446). So the build side for us would be more involved but mostly the same for the vast majority of users. Some users will build from source, but I think that's relatively uncommon/plenty of other libraries are bundling rust already.
I think figuring out the right parts of the api to support rust + pandas + polars would be a bit annoying but doable.
Sounds good. I've certainly noticed Rust pushing it's way into python projects.
Learning Rust is on my 2024 todo list. Unfortunately I can't see that I'll be in a position to offer any help until, at best, towards the end of the year...
No problem! I was going to see if I could carve out a weekend or some time on a flight to try to make a branch.
One thing that doesn't seem to sit right with this library is that it's on a fork. I was wondering whether any move to Rust and support for polars would justify starting over with a fresh library and just crediting the quantopian roots?
Just a thought.
I've definitely been annoyed by that.
I agree that maybe it would be good to develop in a new repo and then archive this one when we're ready (and do a major version update).
Just a dumping ground for ideas.
I think we can extract most of the core logic to rust and then offer functions/packages that produce both polars data and pandas data (via arrow -> pandas conversions in python?).