search

harryprince / geospark

bring sf to spark in production
https://github.com/harryprince/geospark/wiki
57 stars 17 forks source link

Is register_gis compatible with sparkR.session()? #12

Closed MJRichard closed 5 years ago

MJRichard commented 5 years ago

Starting with spark 2.0 the spark connection is created through sparkR.session but the documentation used the older connection.

conf <- spark_config()
sc <- spark_connect(master = "local", config = conf)
register_gis(sc)

Is register_gis() compatible with sparkR.session()?

harryprince commented 5 years ago

@MJRichard do you means SparkR::sparkR.seesion function? currently, sparklyr do not share the session with SparkR, because the session mechanism is different.

The reason that I use sparklyr is it can push down all local R operator to spark, which can reduce the redundant development and bring better consistent user interface.