search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 447 forks source link

Cloudera Livy Kerberos Setup #475

Open sdandey opened 6 years ago

sdandey commented 6 years ago

Can anyone provide instructions on how to setup sparkmagic to communicate with a Kerberoized cloudera cluster?

ericdill commented 6 years ago

Three things are required:

  1. Can you kinit to your Kerberized cluster from the server where you are running a sparkmagic kernel?
  2. Have you configured your sparkmagic config.json to point to the host and port where your Livy server is running?
  3. Is Livy configured correctly to interact with the Kerberized CDH cluster?
sdandey commented 6 years ago

Eric,

see my response below for your questions.

  1. Can you kinit to your Kerberized cluster from the server where you are running a sparkmagic kernel?

Santosh: sparkmagic is installed in developer windows desktop. I can create KDC ticket using the java 8

.

  1. Have you configured your sparkmagic config.json to point to the host and port where your Livy server is running?

Santosh: yes it's pointed to livy server now on the cluster.

  1. Is Livy configured correctly to interact with the Kerberized CDH cluster? `

Santosh: yes it's pointed to interact with kerberized CDH cluster.

`

I'm trying to find-out on how we can implement authentication to Livy Rest API. How can I implement simple http basic authentication to Livy Rest API?