Closed Des135 closed 12 months ago
Q1: Does AWS Glue 4.0 runtime support external jars written & compiled in Java 17? (cause underlying Spark 3.3.0 should support it)
No. Glue 4.0 runtime is Java 8 and it doesn't support external jars written & compiled in Java 17.
Thanks for the answer @aajisaka. Also would be grateful if you could point me to any official AWS docs explicitly mentioning Java 8 as the only supported runtime, cause I wasn't able to find it here https://docs.aws.amazon.com/glue/latest/dg/release-notes.html or in any other related docs
Hi there, I'm trying to run a Glue job locally using glue_libs_4.0.0_image_01 dev container. This Scala script written in Scala 2.12 and references some Java code/classes from an external *.jar provided to the Glue job, written and compiled with Java 17. Glue 4.0 according to documentation runs Spark 3.3.0, which has support of Java 17 https://issues.apache.org/jira/browse/SPARK-33772
When I submit the script+jar to the Docker container, I get the following error in logs:
Question 1: Does AWS Glue 4.0 runtime support external jars written & compiled in Java 17? (cause underlying Spark 3.3.0 should support it) Question 2: If yes, is it possible to update the glue_libs_4.0.0_image_01 dev container to provide Java 17 support?
Thanks