search

mozilla / jupyter-spark

Jupyter Notebook extension for Apache Spark integration
Mozilla Public License 2.0
193 stars 34 forks source link

Handling redirects #45

Closed nbarbosa closed 6 years ago

nbarbosa commented 6 years ago

When connecting to Spark 2.3.0 on a YARN cluster, the UI gets a redirect from port 4040 to 8088. For example

http://localhost:4040 -> http://localhost:8088/proxy/redirect/application_1539820115916_2364/

When I access /spark/api/1/jobs or /spark/api/1/applications it is as though the redirect is not accounted for (shows a Hadoop page)

What would be the best way to handle this?

nbarbosa commented 6 years ago

I got it to work on Yarn with @bernhard-42's fork mentioned here:

https://github.com/mozilla/jupyter-spark/issues/22#issuecomment-385735614