search

acroz / pylivy

A Python client for Apache Livy, enabling use of remote Apache Spark clusters.
MIT License
70 stars 32 forks source link

Upgrade for livy 0.6.0 #56

Open parisni opened 5 years ago

parisni commented 5 years ago

Hi

looks livy 0.6.0 broke the session api

any will to make that compatible ?

acroz commented 5 years ago

Yeah, definitely. I'll look at it when I get a chance.

Do you know the specifics of what they've changed off hand?

acroz commented 4 years ago

Hi @parisni - we are using Livy 0.7.0 in the integration tests. Is there something we're not testing that is broken?

BjoernWaechter commented 3 years ago

Hi we are using livy 0.6.0 installed in a cloudera cluster with kerberos and it seems the proxyUser is not set. The session json looks like this: {"id":13,"name":null,"appId":null,"owner":null,"state":"starting","kind":"pyspark","appInfo":{"driverLogUrl":null,"sparkUiUrl":null},"log":[...]}

zeninpalm commented 3 years ago

Hi @BjoernWaechter @acroz @parisni, I created a PR to resolve this issue and tested it on our cluster, now the code should work as expected.