Doesn't work in Kubernetes

[PoslavskySV]

Hi,

I've created simple k8s pod with almond jupyter:

kind: Pod
apiVersion: v1
metadata:
  name: almond
spec:
  containers:
    - name: almond
      image: almondsh/almond
      ports:
        - name: http
          containerPort: 8888
      resources:
        requests:
          memory: 384Mi
        limits:
          memory: 384Mi
  restartPolicy: Never

so it can be accessible with port forwarding:

kubectl port-forward almond 8888:8888

However, container is always failed to start with the following error:

Container must be run with group "users" to update files
Executing the command: jupyter notebook
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 528, in get
    value = obj._trait_values[self.name]
KeyError: 'runtime_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/jupyter-notebook", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.7/site-packages/jupyter_core/application.py", line 268, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 663, in launch_instance
    app.initialize(argv)
  File "</opt/conda/lib/python3.7/site-packages/decorator.py:decorator-gen-7>", line 2, in initialize
  File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/notebook/notebookapp.py", line 1717, in initialize
    self.init_configurables()
  File "/opt/conda/lib/python3.7/site-packages/notebook/notebookapp.py", line 1372, in init_configurables
    connection_dir=self.runtime_dir,
  File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 556, in __get__
    return self.get(obj, cls)
  File "/opt/conda/lib/python3.7/site-packages/traitlets/traitlets.py", line 535, in get
    value = self._validate(obj, dynamic_default())
  File "/opt/conda/lib/python3.7/site-packages/jupyter_core/application.py", line 99, in _runtime_dir_default
    ensure_dir_exists(rd, mode=0o700)
  File "/opt/conda/lib/python3.7/site-packages/jupyter_core/utils/__init__.py", line 13, in ensure_dir_exists
    os.makedirs(path, mode=mode)
  File "/opt/conda/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.7/os.py", line 211, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/opt/conda/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/home/jovyan/.local'

OK... Specifying security context in pod spec:

...
spec:
  securityContext:
    fsGroup: 0777
  ...

I am able to make it running. But then when connecting to Jupyter, there is no Scala kernel installed there, just only Python 3:

One can check it directly login into pod:

kubectl exec -it almond -- /bin/bash
jovyan@almond:~$ jupyter kernelspec list
Available kernels:
  python3    /opt/conda/share/jupyter/kernels/python3

Can you please help?

Thanks!

[tusharm]

MacOS Catalina 10.15.1 EKS/Kubernetes 1.15 JupyterHub 0.9.0 Almond 0.9.1

Hi,

I had a similar issue where I couldn't use the image in JupyterHub running on Kubernetes (EKS cluster). The kernel wouldn't show up in the list as OP says.

After some investigation, I realized that Almond kernel files are installed inside $HOME. Since JupyterHub's KubeSpawner mounts a volume at $HOME in the container, the kernel files are not visible to Jupyter notebook.

A possible workaround is to run ./almond --global ... while building the image, which results in the kernel getting installed at /usr/local/share/jupyter/.

Thx!

[PoslavskySV]

@tusharm Thanks for the workaround!

[nightscape]

@alexarchambault we just ran into the same problem. Would it be possible to write almond to a different directory by default? If so, shall I create a PR for it?

[JWDobken]

It seams like this issue is fixed. when I reproduce the steps by @PoslavskySV :

$ jupyter kernelspec list
Available kernels:
  scala212    /home/jovyan/.local/share/jupyter/kernels/scala212
  scala213    /home/jovyan/.local/share/jupyter/kernels/scala213
  scala32     /home/jovyan/.local/share/jupyter/kernels/scala32
  python3     /opt/conda/share/jupyter/kernels/python3