Open exsell-jc opened 2 years ago
sdanielzafar
commented
2 years ago Hey there @exsell-jc , this project was put on pause indefinitely because of issues integrating it into the open-source Spark project. I think there could be more to do here, especially surrounding R4 classes you mentioned to provide a PySpark-esque experience for R developers. What did you have in mind?
exsell-jc
commented
2 years ago I didn't have anything specific in mind except maybe a light wrapper for Spark, similar to tidytable. For now, I just wanted some information.
What are some of the issues with integration?
danzafar
commented
2 years ago Just politics, it's a big commit to Spark. If you are interested in writing code we can speak offline, send me an e-mail.
On Tue, Sep 13, 2022 at 1:59 AM exsell-jc @.***> wrote:
I didn't have anything specific in mind except maybe a light wrapper for Spark, similar to tidytable. For now, I just wanted some information.
What are some of the issues with integration?
— Reply to this email directly, view it on GitHub https://github.com/danzafar/tidyspark/issues/74#issuecomment-1245041405, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANUNCJ6IBD26NWU2A5GEBKTV6AX6DANCNFSM5754DAKA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
--
S Daniel Zafar
Resident Solutions Architect
Databricks, Inc.
@.***
281.757.2107
databricks.com
With
tidytable+tidymodelsin R, as well astidypolarsin Python, there is enough justification for 'tidy' to grow more.It seems that this repo is 2 years old, starting with most of the branches not being pruned.
It doesn't seem to support R 4's native pipes
|>either.And it would be really nice if it's on CRAN.
Please let me know if you don't plan on returning to this repo. Though I lack the knowledge, at least only then I will fork it and try something.