I am attempting to run the following in my local PySpark console...
from awsglue.context import GlueContext
glueContext = GlueContext(sc)
We receive the following:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\XYZ\bin\aws-glue-libs\PyGlue.zip\awsglue\context.py", line 47, in __init__
File "C:\Users\XYZ\bin\aws-glue-libs\PyGlue.zip\awsglue\context.py", line 68, in _get_glue_scala_context
TypeError: 'JavaPackage' object is not callable
The following is the complete picture:
The environment looks like the following:
OS: 10.0.17134.0
Python: 3.7.3
Hadoop (winutils.exe): 2.8.5
Spark: 2.4.3
PySpark: 2.4.6
awsglue: 1.0
My environment variables look like the following...
SPARK_HOME: \bin\spark-2.4.3-bin-hadoop2.8\
SPARK_CONF_DIR: \bin\aws-glue-libs\conf\
HADOOP_HOME: \bin\hadoop-2.8.5\
SPARK_CONF_DIR: \bin\spark-2.4.3-bin-hadoop2.8\
JAVA_HOME: C:\Progra~2\Java\jdk1.8.0\
CLASSPATH:
\bin\aws-glue-libs\jarsv1*
\bin\spark-2.4.3-bin-hadoop2.8\jars*
PYTHONPATH:
${SPARK_HOME}\python\lib\py4j
\bin\aws-glue-libs\PyGlue.zip
Just to confirm which version awsglue repo I'm working with...
The following are the "netty" files in my ..\aws-glue-libs\jarsv1\:
I'm looking for a little guidance on how to tweak my configuration to resolve this issue.
I'm having the issue described in issue #42.
I am attempting to run the following in my local PySpark console...
We receive the following:
The following is the complete picture:
The environment looks like the following:
My environment variables look like the following...
Just to confirm which version awsglue repo I'm working with...
The following are the "netty" files in my
..\aws-glue-libs\jarsv1\
:I'm looking for a little guidance on how to tweak my configuration to resolve this issue.