-
Livy 0.5 has a new feature that enables multiple interpreter support in one session:
https://github.com/apache/incubator-livy/commit/c1aafeb6cb87f2bd7f4cb7cf538822b59fb34a9c
Documentation: htt…
ying1 updated
6 years ago
-
I create session and download resulting dataframe. The running code itself is not important:
```python
with LivySession.create(self.LIVY_URL, kind=SessionKind.PYSPARK, requests_session=self.requests…
-
When am trying to connect to sparklyr from r-studio am facing this error
R-Studio installed in remote desktop and trying to connect spark on hdp3.0 cluster.
Generated kerberos ticket but even thoug…
-
I'm trying to access a remote spark cluster using livy, but it seems that the spark context being created is `local` and not the `spark://` cluster address. I have `$SPARK_HOME/conf/slaves` listing th…
-
**Describe the bug**
The pyspark kernel worked fine in Conda jupyter notebook, but it is not showing in the vs code jupyter kernel list. But I can use %%spark to connect my livy service and run code …
-
Hi,
We cannot downgrade livy to 0.7.x without losing Spark 3.x support in livy.
We can still fall back to deliver an early 0.8.0-0.0, 0.8.0-1.0 and resync if/when apache releases 0.8.0
we can a…
-
The 4th container mask isn't currently supported, as the property does not get mapped to a Spark configuration. Specifically:
https://github.com/jahstreet/spark-on-kubernetes-docker/blob/1dc45d3053…
-
I am using MLflow (v 1.13.1) on AWS. I set up a tracking server on a Spark EMR master node then access EMR from SageMaker notebook instance by using Livy. Since I am developing models in Scala Spark, …
-
Would like to enable basic authorization that was configurable at run time. When deploying this a multi-tenant model (ie. each tenant has their own Livy orchestration), we need to secure the authentic…
-
I am aware of the feature that communicate the contents of a dataframe/pandas from livy to local #333 . What about the communication of other basic data structures, like scalars, lists, etc?