Error loading server extension

[mhebrard]

Hi, I am using JupyterHub on AWS EMR v5.24 JupyterHub is installed inside a docker container from AWS. It comes with:

• jupyter_client v5.2.3
• jupyter_core v4.4.0
• jupyterhub v0.9.6

I try to install sparkmonitor within the container as follow

sudo docker exec jupyterhub bash -c "pip install sparkmonitor"
sudo docker exec jupyterhub bash -c "jupyter nbextension install sparkmonitor --py --user --symlink"
sudo docker exec jupyterhub bash -c "jupyter nbextension enable sparkmonitor --py --user"
sudo docker exec jupyterhub bash -c "jupyter serverextension enable --py --user sparkmonitor"

I get message - Validating: OK for the two nbextention lines but after the last line I get the error message:

Enabling: sparkmonitor.serverextension
- Writing config: /home/jovyan/.jupyter
    - Validating...
Error loading server extension sparkmonitor.serverextension
      X is sparkmonitor.serverextension importable?

I continue with

sudo docker exec jupyterhub bash -c "echo \"c.InteractiveShellApp.extensions.append('sparkmonitor.kernelextension')\" >>  /home/jovyan/.ipython/profile_default/ipython_kernel_config.py"

When I run JupyterHub, I see a button "Toggle Spark Monitoring Displays" but I cannot see the monitoring module... any Idea ?

[piorek]

Same here. According to https://github.com/krishnan-r/sparkmonitor/issues/16 running

pip install spark-monitor-s==0.0.11

should help but when running this I'm getting

pip install spark-monitor-s==0.0.11
Collecting spark-monitor-s==0.0.11
  ERROR: Could not find a version that satisfies the requirement spark-monitor-s==0.0.11 (from versions: none)
ERROR: No matching distribution found for spark-monitor-s==0.0.11

[piorek]

it turned out it's sparkonitor-s not spark-monitor-s

[mhebrard]

I use sparkmonitor-s and that solve the error previously mentioned.

sudo docker exec jupyterhub bash -c "pip install sparkmonitor-s==0.0.11"
sudo docker exec jupyterhub bash -c "jupyter nbextension install sparkmonitor --py --user --symlink"
sudo docker exec jupyterhub bash -c "jupyter nbextension enable sparkmonitor --py --user"
sudo docker exec jupyterhub bash -c "jupyter serverextension enable --py --user sparkmonitor"
sudo docker exec jupyterhub bash -c "ipython profile locate default"
sudo docker exec jupyterhub bash -c "echo \"c.InteractiveShellApp.extensions.append('sparkmonitor.kernelextension')\" >>  /home/jovyan/.ipython/profile_default/ipython_kernel_config.py"

When I connect to jupyterHub and create a new PySpark3 notebook, I have a new button in the top menu labelled Toggle Spark Monitor Display

But the module do not appears when I run a job...

[perrital]

"But the module do not appears when I run a job..." did you resolve this issue @mhebrard ?

[dciangot]

yeah, same here. The problem looks like the import in module.js from the browser debugger:

Cannot use import statement outside a module modules.js?v=20200513180012:7

[mhebrard]

I have some issues with other libraries, so I switch to zeppelin, that match my current needs... I might look back to jupyter later on.