search

eakmanrq / sqlframe

Turning PySpark Into a Universal DataFrame API
https://sqlframe.readthedocs.io/en/stable/
MIT License
270 stars 8 forks source link

Support for Spark Java API #132

Closed buremba closed 1 month ago

buremba commented 1 month ago

Thanks for creating this amazing library! It seems like SQLFrame focuses only on PySpark but I'm looking for such library for our codebase, that uses Spark's Java/Scala API. Is this on your radar and if not, can you suggest any alternative if such tooling exists?

eakmanrq commented 1 month ago

Yes you are correct that SQLFrame is just for PySpark. Although I think having this support for the JVM would be cool, it would be a significant investment and I want to stay focused on PySpark for now.

I'm not aware of any tooling that will do this for JVM languages.

Will let you know if I come across something or if I end up deciding to work on JVM support.

Thanks for the feedback!