search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
142 stars 65 forks source link

java.lang.NoSuchMethodError: 'void com.amazonaws.util.IOUtils.closeQuietly #2907 #97

Closed umesh2611 closed 1 year ago

umesh2611 commented 1 year ago

Hi,

I am following the link : https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html to send a kafka message to the MSK cluster ( provisioned / IAM )

When i am sending the message i get the following error :

2023-01-10T14:05:04.035+05:30

Copy 2023-01-10 08:35:04,033 - ERROR - kafka-producer-network-thread | producer-1 - org.apache.kafka.common.utils.KafkaThread - Uncaught exception in thread 'kafka-producer-network-thread | producer-1':

2023-01-10T14:05:04.035+05:30 java.lang.NoSuchMethodError: 'void com.amazonaws.util.IOUtils.closeQuietly(java.io.Closeable, org.apache.commons.logging.Log)' at software.amazon.msk.auth.iam.internals.UserAgentUtils.getLibraryVersion(UserAgentUtils.java:73) at software.amazon.msk.auth.iam.internals.UserAgentUtils.getLibraryVersion(UserAgentUtils.java:73) at software.amazon.msk.auth.iam.internals.UserAgentUtils.(UserAgentUtils.java:43) at software.amazon.msk.auth.iam.internals.UserAgentUtils.(UserAgentUtils.java:43)

complete log :

https://eu-west-1.console.aws.amazon.com/cloudwatch/home?region=eu-west-1#logsV2:log-groups/log-group/$252Faws$252Fecs$252FTmf628NormalizationUPALIWAL/log-events/ecs$252Ftmf628NormalizationContainer2$252F798843bb106f4e5a8fe41e423673e0a5

What i analyzed, the IOUtils pacakged in com.amazonaws.aws-java-sdk-bundle-1.12.262.jar contains a method with the following signature :

public static void com.amazonaws.util.IOUtils.closeQuietly(java.io.Closeable,com.amazonaws.thirdparty.apache.logging.Log)

but "aws-msk-iam-auth" is expecting a method with the following signature : 'void com.amazonaws.util.IOUtils.closeQuietly(java.io.Closeable, org.apache.commons.logging.Log)'

Can you pease help me on moving ahead ?

/Umesh Paliwal

sangeethsasidharan commented 1 year ago

even i am facing same issue , I trying to connect to msk cluster from spark application , Is there any resultion u r able to make here ? @umesh2611

umesh2611 commented 1 year ago

even i am facing same issue , I trying to connect to msk cluster from spark application , Is there any resultion u r able to make here ? @umesh2611

I was able to resolve it. the problem in out case was that the "hadoop-aws" library was loading a sdk-bundle which was of java1 version. I had to exclude the bundle and add the specific libraries.

You will have analyse the dependency tree and check which library is loading an older sdk version of aws and correct it accordingly