search

microsoft / Spark-Hive-Tools

This is for issue/feedback tracking on Spark & Hive Tools
Creative Commons Attribution 4.0 International
6 stars 9 forks source link

SparkMagic v.0.12.9 pyspark3kernel #6

Closed ffung closed 4 years ago

ffung commented 5 years ago

sparkmagic v0.12.9 has removed the obsolete pyspark3kernel https://github.com/jupyter-incubator/sparkmagic/pull/549

The extension now fails when python3 is used.

[2019-8-22:13:5:42] [Info] Version: 0.12.9
Summary: SparkMagic: Spark execution via Livy
Home-page: https://github.com/jupyter-incubator/sparkmagic
Author: Jupyter Development Team
Author-email: jupyter@googlegroups.org
License: BSD 3-clause
Location: /Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages
Requires: requests-kerberos, mock, ipykernel, hdijupyterutils, pandas, ipywidgets, ipython, tornado, notebook, nose, requests, numpy, autovizwidget
Required-by: 

[2019-8-22:13:5:42] [Info] Exec /Users/me/.msvscode.hdinsight/hdinsightJupyter/bin/jupyter, with args: nbextension,enable,--py,--sys-prefix,widgetsnbextension
[2019-8-22:13:5:42] [Info] Enabling notebook extension jupyter-js-widgets/extension...

[2019-8-22:13:5:42] [Info]       - Validating: OK

[2019-8-22:13:5:42] [Info] Exec /Users/me/.msvscode.hdinsight/hdinsightJupyter/bin/jupyter, with args: kernelspec,install,--prefix=/Users/me/.msvscode.hdinsight/hdinsightJupyter,/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/sparkmagic/kernels/pysparkkernel
.[2019-8-22:13:5:42] [Info] [InstallKernelSpec] Removing existing kernelspec in /Users/me/.msvscode.hdinsight/hdinsightJupyter/share/jupyter/kernels/pysparkkernel

[2019-8-22:13:5:42] [Info] [InstallKernelSpec] Installed kernelspec pysparkkernel in /Users/me/.msvscode.hdinsight/hdinsightJupyter/share/jupyter/kernels/pysparkkernel

[2019-8-22:13:5:42] [Info] Exec /Users/me/.msvscode.hdinsight/hdinsightJupyter/bin/jupyter, with args: kernelspec,install,--prefix=/Users/me/.msvscode.hdinsight/hdinsightJupyter,/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/sparkmagic/kernels/pyspark3kernel
[2019-8-22:13:5:43] [Info] Traceback (most recent call last):
  File "/Users/me/.msvscode.hdinsight/hdinsightJupyter/bin/jupyter-kernelspec", line 10, in <module>
    sys.exit(KernelSpecApp.launch_instance())
  File "/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance

[2019-8-22:13:5:43] [Info]     app.start()
  File "/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/jupyter_client/kernelspecapp.py", line 273, in start
    return self.subapp.start()
  File "/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/jupyter_client/kernelspecapp.py", line 143, in start
    replace=self.replace,
  File "/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/jupyter_client/kernelspec.py", line 346, in install_kernel_spec
    shutil.copytree(source_dir, destination)
  File "/Users/me/.pyenv/versions/2.7.15/lib/python2.7/shutil.py", line 185, in copytree
    names = os.listdir(src)
OSError: [Errno 2] No such file or directory: '/Users/me/.msvscode.hdinsight/hdinsightJupyter/lib/python2.7/site-packages/sparkmagic/kernels/pyspark3kernel'
zesluo commented 4 years ago

It has been fixed. Please try the latest version. Thanks for filing the issue