search

krishnan-r / sparkmonitor

Monitor Apache Spark from Jupyter Notebook
https://krishnan-r.github.io/sparkmonitor/
Apache License 2.0
172 stars 55 forks source link

show to configure SPARKMONITOR_KERNEL_PORT #5

Closed wangqiaoshi closed 6 years ago

wangqiaoshi commented 6 years ago

how to configure SPARKMONITOR_KERNEL_PORT

SPARKLISTENER: Exception creating socket:java.lang.NumberFormatException: For input string: "ERRORNOTFOUND"

krishnan-r commented 6 years ago

SPARKMONITOR_KERNEL_PORT is an environment variable set by the IPython kernel extension here, which is used by the scala listener that collects data here. The scala code sends data through a socket to the kernel using this port.

The error maybe because the kernel extension has an error or has not started, so the environement variable is unset.

The kernel extension writes logs to a file in the same folder as the notebook, see if this file is created, and check if the kernel extension is loaded.

Instructions to configure each component, can be found here.

wangqiaoshi commented 6 years ago

kernel extension Has not started.