ClassNotFoundException for JacksonUtils

[moritzvieli]

If I try to produce messages for Kafka with Avro format, I run into this error: java.lang.ClassNotFoundException: software.amazon.awssdk.core.util.json.JacksonUtils

This is my POM:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.2</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>xxx</groupId>
    <artifactId>xxx</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>xxx</name>
    <description>Demo project for Spring Boot</description>
    <properties>
        <java.version>14</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <!-- Kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.glue</groupId>
            <artifactId>schema-registry-serde</artifactId>
            <version>1.1.12</version>
        </dependency>
        <dependency>
            <groupId>software.amazon.msk</groupId>
            <artifactId>aws-msk-iam-auth</artifactId>
            <version>1.1.0</version>
        </dependency>

        <!-- HTTP client -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
        </dependency>

        <!-- XML support -->
        <dependency>
            <groupId>javax.xml.bind</groupId>
            <artifactId>jaxb-api</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>com.sun.xml.bind</groupId>
            <artifactId>jaxb-impl</artifactId>
            <version>2.3.4</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>1.10.2</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/src/main/resources/avro/</sourceDirectory>
                            <outputDirectory>${project.basedir}/src/main/java/models/</outputDirectory>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

Is there anything I am missing?

[moritzvieli]

It seems to be a dependency conflict between schema-registry-serde and aws-msk-iam-auth. Which versions would be compatible? I tried different versions and even added software.amazon.awssdk.utils manually but each version throws a different NoSuchFieldError, NoSuchMethodError or ClassNotFoundException.

[moritzvieli]

Similar issue reported here: https://github.com/aws/aws-msk-iam-auth/issues/62

As those two dependencies are often used together, I guess, there should be an officially supported configuration.

[moritzvieli]

This workaround helps:

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>utils</artifactId>
                <version>2.17.252</version>
            </dependency>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>aws-java-sdk-sts</artifactId>
                <version>1.11.986</version>
            </dependency>
            <dependency>
                <groupId>software.amazon.awssdk</groupId>
                <artifactId>sts</artifactId>
                <version>2.17.192</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

[shaarif-khan]

you can exclude the awssdk util dependency from aws-msk-iam-auth dependency, just like this

software.amazon.msk aws-msk-iam-auth 1.1.4 software.amazon.awssdk utils software.amazon.awssdk sdk-core

[moritzvieli]

@shaarif-khan This does not work either:

com.google.common.util.concurrent.ExecutionError: java.lang.NoSuchMethodError: 'java.util.Optional software.amazon.awssdk.utils.SystemSetting.getStringValueFromEnvironmentVariable(java.lang.String)'