search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
145 stars 67 forks source link

Unsupported callback type:software.amazon.msk.auth.iam.internals.AWSCredentialsCallback when running two wars with msk-auth dependency inside Tomcat #77

Closed IharReznichenka closed 1 year ago

IharReznichenka commented 2 years ago

Hello,

There are some similar tickets already created but the context is a bit different here.

2022-08-10 07:18:36,209 ERROR [SessionID=] [org.apache.kafka.clients.NetworkClient] (kafka-producer-network-thread | intgr-pim-app-workhuman-product-producer) [Producer clientId=producer-01] Connection to node -1 (<>) failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: Exception while evaluating challenge [Caused by javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback type: class: software.amazon.msk.auth.iam.internals.AWSCredentialsCallback classloader: ParallelWebappClassLoader context: app1 delegate: false ----------> Parent Classloader: java.net.URLClassLoader@75a1cd57 from class: software.amazon.msk.auth.iam.IAMClientCallbackHandlerclassloader: ParallelWebappClassLoader context: app2 delegate: false ----------> Parent Classloader: java.net.URLClassLoader@75a1cd57 ]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state.

It seems that the issue here is the class software.amazon.msk.auth.iam.internals.AWSCredentialsCallback was previously loaded by the app1 and can't be used by the app2. It's a bit confusing why app2 tries to use the class loaded by app1 taking into consideration that they have different contexts.

The setup is following:

Configuration is the same for both apps except the profile name

kafka.producer.sasl.mechanism=AWS_MSK_IAM kafka.producer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="profile-name"; kafka.producer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

It works fine when there is only one app deployed that uses aws-msk-iam-auth. But as soon as there are two apps, the first one loads fine but during the deployment of the second app, we start seeing the exception above.

Can you please advise if there is something wrong with the configuration or should we meet some specific deployment requirement to be able to use the lib?

Please let me know if any additional information is required here.

IharReznichenka commented 2 years ago

I was able to make it work (details below), but such a ClassLoader setup is not an option in our case because it brings a lot of unnecessary dependencies to other webapps. It would be great to hear your idea if there are other options.

The main idea is to use the Shared ClassLoader - https://tomcat.apache.org/tomcat-9.0-doc/class-loader-howto.html

conf/catalina.properties

shared.loader="${catalina.base}/shared-lib-3rd-party"

shared-lib-3rd-party

aws-msk-iam-auth-jar-with-dependencies.jar

pom.xml

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.amazonaws</groupId>
                <artifactId>aws-java-sdk-bom</artifactId>
                <version>1.11.986</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>bom</artifactId>
                <version>2.17.122</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-core</artifactId>
        </dependency>
        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-sts</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>auth</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>3.2.1</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.msk</groupId>
            <artifactId>aws-msk-iam-auth</artifactId>
            <version>1.1.4</version>
            <exclusions>
                <exclusion>
                    <groupId>*</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.4.2</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                        <configuration>
                            <descriptorRefs>
                                <descriptorRef>jar-with-dependencies</descriptorRef>
                            </descriptorRefs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
dannycranmer commented 2 years ago

This is the same root cause as https://github.com/aws/aws-msk-iam-auth/issues/36

github-actions[bot] commented 1 year ago

We have noticed this issue has not received attention in 1 year. We will close this issue for now. If you think this is in error, please feel free to comment and reopen the issue.