search

jupyter-incubator / sparkmagic

Jupyter magics and kernels for working with remote Spark clusters
Other
1.33k stars 447 forks source link

Unable to start Spark or PySpark kernels - No module named winkerberos #503

Open Nithanaroy opened 5 years ago

Nithanaroy commented 5 years ago

On creating a new notebook with either Spark or PySpark kernels on a CentOS 7 machine, this error appears and kernel never starts,

/export/apps/python/2.7/bin/python: No module named winkerberos
[I 11:16:54.163 NotebookApp] KernelRestarter: restarting kernel (1/5), new random ports
/export/apps/python/2.7/bin/python: No module named winkerberos
[I 11:16:57.171 NotebookApp] KernelRestarter: restarting kernel (2/5), new random ports
/export/apps/python/2.7/bin/python: No module named winkerberos
[I 11:17:00.181 NotebookApp] KernelRestarter: restarting kernel (3/5), new random ports
/export/apps/python/2.7/bin/python: No module named winkerberos
[I 11:17:03.189 NotebookApp] KernelRestarter: restarting kernel (4/5), new random ports
/export/apps/python/2.7/bin/python: No module named winkerberos
[W 11:17:06.198 NotebookApp] KernelRestarter: restart failed
[W 11:17:06.199 NotebookApp] Kernel 82b4f257-29f9-46bb-82db-4057701ff819 died, removing from map.

I'm not sure if winkerberos can be installed on a linux machine though it was mentioned as an optional step in the README.

Nithanaroy commented 5 years ago

These libraries are already installed,

installed pykerberos using conda before pip install sparkmagic

ktmud commented 5 years ago

The error comes from import kerberos, and is because of incompatible version with Python 2.

Try this:

pip install pykerberos==1.1.14
MarkusShepherd commented 5 years ago

I'm running into the same issue on Mac. Pinning pykerberos to v1.1.14 seems to fix it, but this seems like it needs to be solved by updating dependencies in setup.py.

itamarst commented 5 years ago

FYI Spark will be dropping Python 2 support soon, and so will Sparkmagic probably. I strongly recommend migrating to Python 3.

MarkusShepherd commented 5 years ago

I encountered the same problem in Python 3, so this is not a pure Python 2 issue.

itamarst commented 5 years ago

Good to know, thanks.

itamarst commented 5 years ago

I can't reproduce this on Python 3. Can you @MarkusShepherd :

  1. Include the output of pip list.
  2. Tell me exactly what you did to get the error.

Thanks!

duynguyenhoang commented 4 years ago

I had similar problem with my Python 3.7 conda environment Try to run conda install pykerberos will solve the issue.

sudi007 commented 4 years ago

@itamarst i having the same issue can you guide me what i can do fix it please and below is the error i get

/opt/anaconda3/envs/jupyter-hub/bin/python: Error while finding module specification for 'sparkmagi c.kernels.pysparkkernel.pysparkkernel' (ModuleNotFoundError: No module named 'sparkmagic') [I 2020-05-27 23:11:32.180 SingleUserNotebookApp restarter:110] KernelRestarter: restarting kernel (1/5), new random ports /opt/anaconda3/envs/jupyter-hub/bin/python: Error while finding module specification for 'sparkmagi c.kernels.pysparkkernel.pysparkkernel' (ModuleNotFoundError: No module named 'sparkmagic') [I 2020-05-27 23:11:35.186 SingleUserNotebookApp restarter:110] KernelRestarter: restarting kernel (2/5), new random ports /opt/anaconda3/envs/jupyter-hub/bin/python: Error while finding module specification for 'sparkmagi c.kernels.pysparkkernel.pysparkkernel' (ModuleNotFoundError: No module named 'sparkmagic') [I 2020-05-27 23:11:38.193 SingleUserNotebookApp restarter:110] KernelRestarter: restarting kernel (3/5), new random ports /opt/anaconda3/envs/jupyter-hub/bin/python: Error while finding module specification for 'sparkmagi c.kernels.pysparkkernel.pysparkkernel' (ModuleNotFoundError: No module named 'sparkmagic') [I 2020-05-27 23:11:41.199 SingleUserNotebookApp restarter:110] KernelRestarter: restarting kernel (4/5), new random ports /opt/anaconda3/envs/jupyter-hub/bin/python: Error while finding module specification for 'sparkmagi c.kernels.pysparkkernel.pysparkkernel' (ModuleNotFoundError: No module named 'sparkmagic') [W 2020-05-27 23:11:44.206 SingleUserNotebookApp restarter:100] KernelRestarter: restart failed [W 2020-05-27 23:11:44.206 SingleUserNotebookApp kernelmanager:135] Kernel 692153f5-7190-41b0-9555- 9945306e47a8 died, removing from map. [W 2020-05-27 23:11:50.228 SingleUserNotebookApp handlers:252] Replacing stale connection: 692153f5 -7190-41b0-9555-9945306e47a8:b618d3511a64404893f37dcdc2b093fc