search

r-dbi / backends

Details on DBI backends
https://www.r-dbi.org/backends/
15 stars 6 forks source link

New DBI backend for Spark #76

Open alibell opened 1 year ago

alibell commented 1 year ago

Hello,

I've created a backend to add Spark connection support using the SparkR library.

There are actually two libraries which permit Spark connection with R: SparkR and Sparklyr. Even if the Sparklyr interface exists, SparkR is the Spark official one and allow the user to follow Spark syntax and perform lazy and distributed computation before data retrieval. I think it would be useful for users working with the SparkR library to have a DBI interface to keep compatibility with many R tools and packages.

My package repository is: https://github.com/alibell/SparkR-DBI

Related PR: https://github.com/r-dbi/backends/pull/77

Thank you

krlmlr commented 1 year ago

Thanks, nice work! This should be picked up automatically once the package is on CRAN.