search

awslabs / aws-glue-libs

AWS Glue Libraries are additions and enhancements to Spark for ETL operations.
Other
635 stars 299 forks source link

class conflict on running glue with spark #184

Open ravive-neosec opened 1 year ago

ravive-neosec commented 1 year ago

org.apache.iceberg:iceberg-spark-runtime-3.1_2.12:1.2.1 software.amazon.awssdk:url-connectionclient:2.20.18, software.amazon.awssdk:sts:2.20.18, software.amazon.awssdk:s3:2.20.18, software.amazon.awssdk:utils:2.20.18

this is the jars im using with glue ver 3.0.0 getting an error

py4j.protocol.Py4JJavaError: An error occurred while calling o87.sql. : java.lang.NoSuchMethodError: 'java.util.Map software.amazon.awssdk.utils.CollectionUtils.uniqueIndex(java.lang.Iterable, java.util.function.Function)' at software.amazon.awssdk.utils.internal.EnumUtils.uniqueIndex(EnumUtils.java:39) at software.amazon.awssdk.services.s3.model.ObjectCannedACL.(ObjectCannedACL.java:40) at org.apache.iceberg.aws.AwsProperties.(AwsProperties.java:910) at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.initialize(AwsClientFactories.java:141) at org.apache.iceberg.aws.AwsClientFactories.loadClientFactory(AwsClientFactories.java:87)