I have a project with glue 4 and the appropriate scala and spark versions according to the AWS documentation.
"com.amazonaws" % "AWSGlueETL" % "4.0.0" % Provided, "com.vitorsvieira" %% "scala-iso" % "0.1.2", "org.apache.logging.log4j" % "log4j-core" % "2.13.1" % Provided, "org.apache.spark" %% "spark-core" % "3.3.2" % Provided, "org.apache.spark" %% "spark-mllib" % "3.3.2" % Provided, "org.apache.spark" %% "spark-sql" % "3.3.2" % Provided,
but I am getting the following error.
Since the error occurred, the method does not receive 2 parameters in the jar (the dependency that is downloaded with the issue's upload date), but when commenting on this error I obtained a jar that I downloaded last April in which that method contains the syntax of the 2 parameters. And in addition to that there is no problem when running it in AWS
As part of my tests I replaced the automatically downloaded (current) jar with the one I had previously downloaded and it worked without problems.
My question is why in AWS it does not download the same one as on my (current) computer, there is a risk that AWS will download the jar that is not compatible and my processes will give errors
I have a project with glue 4 and the appropriate scala and spark versions according to the AWS documentation.
"com.amazonaws" % "AWSGlueETL" % "4.0.0" % Provided, "com.vitorsvieira" %% "scala-iso" % "0.1.2", "org.apache.logging.log4j" % "log4j-core" % "2.13.1" % Provided, "org.apache.spark" %% "spark-core" % "3.3.2" % Provided, "org.apache.spark" %% "spark-mllib" % "3.3.2" % Provided, "org.apache.spark" %% "spark-sql" % "3.3.2" % Provided,but I am getting the following error.Since the error occurred, the method does not receive 2 parameters in the jar (the dependency that is downloaded with the issue's upload date), but when commenting on this error I obtained a jar that I downloaded last April in which that method contains the syntax of the 2 parameters. And in addition to that there is no problem when running it in AWS As part of my tests I replaced the automatically downloaded (current) jar with the one I had previously downloaded and it worked without problems.
My question is why in AWS it does not download the same one as on my (current) computer, there is a risk that AWS will download the jar that is not compatible and my processes will give errors