Open singlewind opened 1 year ago
I don't know if this is the best way to handle this, but just to make the warning go away I changed lines 56-57 of context.py to
def __init__(self, sparkContext, sparkSession=None, **options):
if sparkSession is not None:
super(GlueContext, self).__init__(sparkContext, sparkSession=sparkSession)
else:
super(GlueContext, self).__init__(sparkContext)
and it seemed to work if i passed in the session.
See below warnings for glue 4.0. Just quickly check the code, may be we can pass sparkSession to GlueContext instead of create one.