search

apache-spark-on-k8s / spark

Apache Spark enhanced with native Kubernetes scheduler back-end: NOTE this repository is being ARCHIVED as all new development for the kubernetes scheduler back-end is now on https://github.com/apache/spark/
https://spark.apache.org/
Apache License 2.0
612 stars 118 forks source link

Executors tab not accessible from driver UI #304

Open foxish opened 7 years ago

foxish commented 7 years ago

I'm accessing the driver UI after running kubectl proxy, at URLs that look like: http://localhost:8080/api/v1/namespaces/default/pods/:4040/proxy All tabs except Executors works. The Executors tab shows <h3>unauthorized</h3>.

I suspect it may be rejecting the embedding of the executor page based on the same-origin-policy that may need to specified through spark.ui.allowFramingFrom.

foxish commented 7 years ago

cc @lins05 @ash211

foxish commented 7 years ago

@lins05 @kimoonkim @mccheah will you have time to look at this?

ash211 commented 7 years ago

Seems like a similar problem as https://github.com/apache-spark-on-k8s/spark/pull/95 -- maybe when running through a kubectl proxy those the k8sProxyPattern doesn't match?

foxish commented 7 years ago

Looks like a different problem here. With the v1 server, we used to create a service for the UI that we no longer create. The unauthorized message is coming from the K8s side and not the spark side. I'm not sure why, but it could be some component stripping out the proxy authentication headers.

The problem doesn't exist when the UI is accessed via the service + APIServer proxy. Considering adding the UI service definition to the job.