[Windows] Worker and Driver has diffrent version

dsp-uga / andromeda

This repository contains a Naive Bayes classifier implemented on document classification which is completed on CSCI 8360, Data Science Practicum at the University of Georgia, Spring 2018.

MIT License

4 stars 1 forks source link

[Windows] Worker and Driver has diffrent version #19

Closed nihalsoans91 closed 6 years ago

nihalsoans91 commented 6 years ago

Spark Exception: Python in worker has different version 3.5 than that in driver 3.6

Windows + PyCharm

nihalsoans91 commented 6 years ago

The environment Variable PYSPARK_DRIVER_PYTHON should be set as ipython3

This error is caused when the driver runs from my anaconda environment and worker runs from the inbuilt python available prebuilt in the OS