search

rstudio / sparkapi

Sparklyr Extensions API
http://spark.rstudio.com/extensions.html
Apache License 2.0
31 stars 13 forks source link

Example Extension Package needed #5

Open nicornk opened 8 years ago

nicornk commented 8 years ago

Hi,

I am trying to understand the concepts behind the extensions for sparklyr.

An example extension would be very helpful for me.

Thanks!

jjallaire commented 8 years ago

We'll post a sample soon!

On Tue, Jul 5, 2016 at 10:16 AM, Nicolas Rnkmp notifications@github.com wrote:

Hi,

I am trying to understand the concepts behind the extensions for sparklyr.

An example extension would be very helpful for me.

BR Nicolas

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/rstudio/sparkapi/issues/5, or mute the thread https://github.com/notifications/unsubscribe/AAGXx1aRA_T0KPrj5uk7feeqPTaRYw-_ks5qSmcogaJpZM4JFKeP .

jwijffels commented 8 years ago

I tried this out and created an R package called spark.sas7bdat which imports SAS data in parallel in Spark More information: https://github.com/bnosac/spark.sas7bdat Thanks for the documentation about the usage of the sparkapi package. This worked nicely!

One question. Is it desireable to include spark packages from https://spark-packages.org/ inside the java folder of the R package to reference it from there or is it advised to use the packages argument of the spark_dependency function.

jjallaire commented 8 years ago

That's fantastic!!!!

Yes, you should definitely use the packages argument of the spark_dependency function (as you have done in your example) in preference to embedding within the java folder of the R package.

On Wed, Jul 20, 2016 at 4:56 PM, jwijffels notifications@github.com wrote:

I tried this out and created an R package called spark.sas7bdat which imports SAS data in parallel in Spark More information: https://github.com/bnosac/spark.sas7bdat Thanks for the documentation about the usage of the sparkapi package. This worked nicely!

One question. Is it desireable to include spark packages from https://spark-packages.org/ inside the java folder of the R package to reference it from there or is it advised to use the packages argument of the spark_dependency function.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rstudio/sparkapi/issues/5#issuecomment-234080434, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGXxxm7kZa9pX4LbhcUwILkBsN7qbsSks5qXouHgaJpZM4JFKeP .