search

JahstreetOrg / spark-on-kubernetes-helm

Spark on Kubernetes infrastructure Helm charts repo
Apache License 2.0
199 stars 75 forks source link

Add Delta Lake Support #73

Open lcmarques opened 2 years ago

lcmarques commented 2 years ago

One of the things that it is really nice to have is the delta lake support (https://delta.io/) for Spark. It is wildly used right now (pushed by Databricks). Is there any good way to start looking at it?

jahstreet commented 2 years ago

Hi @lcmarques , soon I plan to get back to data engineering with Spark on a daily basis and plan to become more proactive in supporting the project. In the meantime, I 100% agree with your statement and will be glad to support any initiative around boosting the feature richness of the project. Do you have any specific use-case for it already? Will be glad to assist you along the investigations.