seajosh
commented
7 years ago vagrant@precise64:~$ pyspark
Python 3.6.0 |Continuum Analytics, Inc.| (default, Dec 23 2016, 12:22:00)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
File "/home/vagrant/spark/python/pyspark/shell.py", line 30, in <module>
import pyspark
File "/home/vagrant/spark/python/pyspark/__init__.py", line 44, in <module>
from pyspark.context import SparkContext
File "/home/vagrant/spark/python/pyspark/context.py", line 36, in <module>
from pyspark.java_gateway import launch_gateway
File "/home/vagrant/spark/python/pyspark/java_gateway.py", line 31, in <module>
from py4j.java_gateway import java_import, JavaGateway, GatewayClient
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
File "/home/vagrant/spark/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 18, in <module>
File "/home/vagrant/anaconda/lib/python3.6/pydoc.py", line 62, in <module>
import pkgutil
File "/home/vagrant/anaconda/lib/python3.6/pkgutil.py", line 22, in <module>
ModuleInfo = namedtuple('ModuleInfo', 'module_finder name ispkg')
File "/home/vagrant/spark/python/pyspark/serializers.py", line 393, in namedtuple
cls = _old_namedtuple(*args, **kwargs)
TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'
>>>
rjurney
Pyspark doesn't work with Python 3.6.0
https://issues.apache.org/jira/browse/SPARK-19019