search

sparklyr / sparklyr

R interface for Apache Spark
https://spark.rstudio.com/
Apache License 2.0
955 stars 309 forks source link

Livy support for Spark 3.x #3164

Open yitao-li opened 3 years ago

yitao-li commented 3 years ago

Context: https://github.com/sparklyr/sparklyr/issues/3162

Probably not much work here other than verifying existing jar works with Livy

yitao-li commented 3 years ago

Note to myself: according to https://issues.apache.org/jira/browse/LIVY-756 I need to wait until Livy 0.8 is officially released before this can happen.

harryprince commented 2 years ago

livy connection is pretty slow and inefficient, recommend connecting by yarn

bgajjela commented 3 months ago

Looks like livy-0.8.0 is released. https://github.com/apache/incubator-livy/releases/tag/v0.8.0-incubating