yutannihilation
commented
1 month ago Oh, thanks for catching! I didn't notice there are such prior works. Looks useful.
The main purpose of this repo is to see if my framework has enough features for mildly complex usages. My focus is
- conversion from external pointer or ALTREP that other package defines. e.g. https://arrow.apache.org/datafusion-python/
- async support e.g. https://github.com/yutannihilation/savvy-async-poc
I'm not sure if this will eventually become a serious project, but I'll consider donating to the upstream.
I know this is a work in progress, I just wanted to say that I think it's great!
I am sure you've seen https://josiahparry.com/posts/2023-11-24-dfusionrdr/#source-code , but wanted to put it here in case you hadn't yet!
At one point I made https://github.com/paleolimbot/datafusion but it was based on a C wrapper around datafusion that somebody had put together and is perhaps not all that useful.
If you're ever interested in donating this to apache/datafusion I'm happy to help with the Apache details (or general Arrow details if you think I could be useful!)