-
If sparklyr had the equivalent or similar to `SparkR::dapply` then we could run R user defined functions on rows of data. This would allow using package like `lubridate` at scale and build a very val…
-
How would we do something similar to the apply functions in SparkR?
``` r
dapply(x, func, schema)
spark.lapply(sc, list, func)
```
[Documentation can be found here](http://spark.apache.org/docs/2.0.…
-
I need to read and write to an hdfs cluster where the format of files is "avro"
Is there any way to get arround the standard functions to do it ?
It would be great to have a spark_read_avro and sp…
-
Hi, when I setup sparklyr on windows 10 as described in the RStudio tutorial
```
>devtools::install_github("rstudio/sparkapi", force = TRUE)
Downloading GitHub repo rstudio/sparkapi@master
from URL …
-
I am having a problem with the latest version (0.4.15) of sparklyr connecting. I had the same issue with spark 1.6.2. Here is the error logs for the two attempts to connect + `sessionInfo`:
## Attem…
-
Hi -
Not sure if this is an issue from my side or not. I am currently trying to use ALS Matrix Factorization [http://spark.rstudio.com/reference/sparklyr/latest/ml_als_factorization.html](url) in s…
-
this is result of sys.getenv(), My sytem is Win 7.
```
> Sys.getenv()
_DFX_INSTALL_UNSIGNED_DRIVER
1
ALLUSERSPROFILE C:\ProgramData
APPDATA C:\Users\Administ…
-
Hi!
I try to load a big csv file into Spark with the spark_read_csv function of Sparklyr. The command takes very long to execute and it finally kills the RStudio session. When I look into the body …
-
From this list: https://github.com/rstudio/sparklyr/blob/master/inst/scala/utils.scala#L130-L135
would make sense to implement all the types mapping as listed here:
https://spark.apache.org/docs/1.6…
-
I am running:
1. Windows 7
2. R 3.3.1
3. RStudio .99.1246
4. Spark 1.6.2 Pre-built for Hadoop 2.6
**When I run the following code:**
install.packages("devtools")
devtools::install_github("rstudio/spa…