search

mlverse / pysparklyr

Extension to {sparklyr} that allows you to interact with Spark & Databricks Connect
https://spark.posit.co/deployment/databricks-connect.html
Other
14 stars 3 forks source link
databricks pyspark r spark spark-connect

readme

pysparklyr

R-CMD-check Spark-Connect codecov CRAN status

Integrates sparklyr with PySpark and Databricks. The main reason of this package is because the new Spark and Databricks Connect connection method does not work with standard sparklyr integration.

Installing

To install the version in CRAN use:

install.packages("pysparklyr")

To get the development version from GitHub use:

remotes::install_github("mlverse/pysparklyr")

Using

To learn how to use, please visit the Spark / Databricks Connect article, available in the official sparklyr website: Spark Connect, and Databricks Connect v2