Closed just-JL closed 4 years ago
The way sparkmagic's SQL works is it sends Python code to PySpark to execute. Not ideal, but that's how it happens now. And part of that code detects the Python version.
However... 'tuple' object has no attribute 'major' shouldn't happen in Python 2.7 or later, only in Python 2.6 or earlier. And PySpark 2.3.1 doesn't use Python 2.6... so it's a little confusing.
Can you do "import sys; print(sys.version_info)" in a normal Python Sparkmagic cell (on the server-side)?
it is (2, 6, 6, 'final', 0)
Yeah, Python 2.6 is not supported (Python 2.7 was released in 2010).
Python 2 support in general is going to end sometime next year, so I strongly recommend starting the switch to Python 3.
The way sparkmagic's SQL works is it sends Python code to PySpark to execute. Not ideal, but that's how it happens now. And part of that code detects the Python version.
However... 'tuple' object has no attribute 'major' shouldn't happen in Python 2.7 or later, only in Python 2.6 or earlier. And PySpark 2.3.1 doesn't use Python 2.6... so it's a little confusing.
Can you do "import sys; print(sys.version_info)" in a normal Python Sparkmagic cell (on the server-side)?
It occured 'tuple' object has no attribute 'major' when I tried to execute datax.py which might depend on PySpark, or not, to complete the setting of dataX.
After I read your comment, I changed my python from 2.6 to 2.7. Then the problem had been solved.
hi, i am facing this question. When i execute this code in PySpark, The session state is idle and the executors of spark ui only has a driver. I am confused. Thanks in advance.