ERROR: test_transform_data (tests.test_etl_job.SparkETLTests)
Test data transformer.
Traceback (most recent call last):
File "/home/brix/pyspark-workloads/pyspark-example-project/tests/test_etljob.py", line 27, in setUp
self.spark, * = start_spark()
File "/home/brix/pyspark-workloads/pyspark-example-project/dependencies/spark.py", line 88, in start_spark
spark_sess = spark_builder.getOrCreate()
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/sql/session.py", line 173, in getOrCreate
sc = SparkContext.getOrCreate(sparkConf)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 349, in getOrCreate
SparkContext(conf=conf or SparkConf())
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 118, in init
conf, jsc, profiler_cls)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 195, in _do_init
self._encryption_enabled = self._jvm.PythonUtils.getEncryptionEnabled(self._jsc)
File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/py4j/java_gateway.py", line 1487, in getattr
"{0}.{1} does not exist in the JVM".format(self._fqn, name))
py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM
ERROR: test_transform_data (tests.test_etl_job.SparkETLTests) Test data transformer.
Traceback (most recent call last): File "/home/brix/pyspark-workloads/pyspark-example-project/tests/test_etljob.py", line 27, in setUp self.spark, * = start_spark() File "/home/brix/pyspark-workloads/pyspark-example-project/dependencies/spark.py", line 88, in start_spark spark_sess = spark_builder.getOrCreate() File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/sql/session.py", line 173, in getOrCreate sc = SparkContext.getOrCreate(sparkConf) File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 349, in getOrCreate SparkContext(conf=conf or SparkConf()) File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 118, in init conf, jsc, profiler_cls) File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/pyspark/context.py", line 195, in _do_init self._encryption_enabled = self._jvm.PythonUtils.getEncryptionEnabled(self._jsc) File "/home/brix/pyspark-workloads/pyspark-example-project/.venv/lib/python3.6/site-packages/py4j/java_gateway.py", line 1487, in getattr "{0}.{1} does not exist in the JVM".format(self._fqn, name)) py4j.protocol.Py4JError: org.apache.spark.api.python.PythonUtils.getEncryptionEnabled does not exist in the JVM