-
see https://github.com/harryprince/geospark/wiki/Introduction-&-Philosophy
-
What do we need to put into the `requirements.txt` file to make the PySpark and sparklyr code run? e.g. pyspark>=2.4.0, sparklyr >=1.7.4
-
+ Bridging python packages (`dask-geopandas`, `geopolars`, `cuSpatial`) to R workflow
+ Optimal splitting of computational regions regarding shape complexity and input data resolutions
+ Streamlined…
-
See https://twitter.com/jsonbecker/status/896118907949334528.
The `aws.s3` package could implement a connection contract and UI that would make S3 connecitons available in RStudio, similar to #spar…
-
The vignette no longer compiles (at least for me). This is happening to others.
Error in spark_connect(master = "local[2]", version = "2.4.3") :
could not find function "spark_connect"
I ha…
-
I'm getting an issue to previous issues reported on here, but the fixes don't seem to solve this one.
```
port = 7078
config
-
I am working with sparkxgb on an Amazon EMR version 5.16 Spark cluster with RStudio 1.1.447 and R version 3.4.1. I install the libraries with:
devtools::install_github("rstudio/sparklyr")
devtools:…
-
# Feature request
Support `org.apache.spark.mllib.random.RandomRDDs.*VectorRDD`
The current implementation of random sample generation in `sparklyr::sdf_r*()` only supports scalar output (ie. si…
-
We recently upgraded our cluster from HDP 2.6 to HDP 3.1, which has
```
Hive 3.1.0.3.1.0.0-78
Spark - 2.3.2.3.1.0.0-78
Hadoop - 3.1.1.3.1.0.0-78
Sparklyr - 0.9.2
```
we had sparklyr code o…
-
related to https://github.com/HenrikBengtsson/future/issues/286, A good idea comes from `future` package.
the future package provides a great parallel framework across multi-engine, including socke…