[BUG] Netty class not found httpClientSecure

[nesanmano]

Describe the bug I am trying to use Azure SDK in my Spring Boot application. I have done put everything correctly except for one thing the netty bom dependency.

I keep getting this every time I run my test.

2021-10-05 16:34:09.556 ERROR --- [onPool-worker-1] c.a.i.ClientCertificateCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Could not initialize class reactor.netty.http.client.HttpClientSecure 2021-10-05 16:34:09.582 ERROR --- [onPool-worker-1] c.a.c.implementation.AccessTokenCache : Failed to acquire a new access token. 2021-10-05 16:34:39.589 ERROR --- [onPool-worker-3] c.a.i.ClientCertificateCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Could not initialize class reactor.netty.http.client.HttpClientSecure 2021-10-05 16:34:39.591 ERROR --- [onPool-worker-3] c.a.c.implementation.AccessTokenCache : Failed to acquire a new access token. 2021-10-05 16:35:09.601 ERROR --- [onPool-worker-5] c.a.i.ClientCertificateCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Could not initialize class reactor.netty.http.client.HttpClientSecure 2021-10-05 16:35:09.602 ERROR --- [onPool-worker-5] c.a.c.implementation.AccessTokenCache : Failed to acquire a new access token. 2021-10-05 16:35:39.610 ERROR --- [onPool-worker-7] c.a.i.ClientCertificateCredential : Azure Identity => ERROR in getToken() call for scopes [https://management.core.windows.net//.default]: Could not initialize class reactor.netty.http.client.HttpClientSecure 2021-10-05 16:35:39.611 ERROR --- [onPool-worker-7] c.a.c.implementation.AccessTokenCache : Failed to acquire a new access token. 2021-10-05 16:35:39.611 INFO --- [onPool-worker-7] com.azure.core.http.policy.RetryPolicy : Retry attempts have been exhausted after 3 attempts.

java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientSecure

at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.lambda$subscribe$0(HttpClientConnect.java:219)
at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57)
at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:77)
at reactor.core.publisher.MonoRetryWhen.subscribeOrReturn(MonoRetryWhen.java:46)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.netty.http.client.HttpClientConnect$MonoHttpConnect.subscribe(HttpClientConnect.java:271)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)
at reactor.core.publisher.FluxRetryWhen.subscribe(FluxRetryWhen.java:77)
at reactor.core.publisher.MonoRetryWhen.subscribeOrReturn(MonoRetryWhen.java:46)
at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:57)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52)
at reactor.core.publisher.Mono.subscribe(Mono.java:4338)
at reactor.core.publisher.Mono.block(Mono.java:1703)
at com.azure.identity.implementation.HttpPipelineAdapter.send(HttpPipelineAdapter.java:58)
at com.microsoft.aad.msal4j.HttpHelper.executeHttpRequestWithRetries(HttpHelper.java:96)
at com.microsoft.aad.msal4j.HttpHelper.executeHttpRequest(HttpHelper.java:49)
at com.microsoft.aad.msal4j.AadInstanceDiscoveryProvider.executeRequest(AadInstanceDiscoveryProvider.java:232)
at com.microsoft.aad.msal4j.AadInstanceDiscoveryProvider.sendInstanceDiscoveryRequest(AadInstanceDiscoveryProvider.java:187)
at com.microsoft.aad.msal4j.AadInstanceDiscoveryProvider.doInstanceDiscoveryAndCache(AadInstanceDiscoveryProvider.java:285)
at com.microsoft.aad.msal4j.AadInstanceDiscoveryProvider.getMetadataEntry(AadInstanceDiscoveryProvider.java:57)
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.getAuthorityWithPrefNetworkHost(AuthenticationResultSupplier.java:39)
at com.microsoft.aad.msal4j.AcquireTokenSilentSupplier.execute(AcquireTokenSilentSupplier.java:23)
at com.microsoft.aad.msal4j.AcquireTokenByClientCredentialSupplier.execute(AcquireTokenByClientCredentialSupplier.java:46)
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:69)
at com.microsoft.aad.msal4j.AuthenticationResultSupplier.get(AuthenticationResultSupplier.java:18)
at java.util.concurrent.CompletableFuture$AsyncSupply.run$$$capture(CompletableFuture.java:1604)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java)
at java.util.concurrent.CompletableFuture$AsyncSupply.exec(CompletableFuture.java:1596)
at java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

This is a part of my POM dependency file:

    <dependency>
        <groupId>com.clickntap</groupId>
        <artifactId>vimeo</artifactId>
        <version>1.13</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-administration</artifactId>
        <version>4.0.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-certificates</artifactId>
        <version>4.2.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-jca</artifactId>
        <version>2.0.0</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-keys</artifactId>
        <version>4.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-secrets</artifactId>
        <version>4.3.3</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core-management</artifactId>
        <version>1.4.1</version>
    </dependency>

    <dependency>
        <groupId>com.azure.resourcemanager</groupId>
        <artifactId>azure-resourcemanager-resources</artifactId>
        <version>2.8.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure.resourcemanager</groupId>
        <artifactId>azure-resourcemanager-keyvault</artifactId>
        <version>2.8.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-keys</artifactId>
        <version>4.3.2</version>
    </dependency>

    <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>msal4j</artifactId>
        <version>1.11.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-administration</artifactId>
        <version>4.0.3</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.3.6</version>
        <exclusions>
            <exclusion>
                <groupId>io.projectreactor</groupId>
                <artifactId>reactor-core</artifactId>
            </exclusion>
            <exclusion>
                <groupId>io.projectreactor.netty</groupId>
                <artifactId>reactor-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-tcnative-boringssl-static</artifactId>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core</artifactId>
        <version>1.20.0</version>
    </dependency>

    <!-- Reactor -->
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-core</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-test</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-tools</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-extra</artifactId>
        <version>3.4.4</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-adapter</artifactId>
        <version>3.4.4</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.netty</groupId>
        <artifactId>reactor-netty</artifactId>
        <version>1.0.10</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.netty</groupId>
        <artifactId>reactor-netty-core</artifactId>
        <version>1.0.10</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.netty</groupId>
        <artifactId>reactor-netty-http</artifactId>
        <version>1.0.10</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.netty</groupId>
        <artifactId>reactor-netty-http-brave</artifactId>
        <version>1.0.10</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-pool</artifactId>
        <version>0.2.6</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.kafka</groupId>
        <artifactId>reactor-kafka</artifactId>
        <version>1.3.5</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.rabbitmq</groupId>
        <artifactId>reactor-rabbitmq</artifactId>
        <version>1.5.3</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.kotlin</groupId>
        <artifactId>reactor-kotlin-extensions</artifactId>
        <version>1.1.4</version>
    </dependency>

    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-handler</artifactId>
        <version>4.1.67.Final</version>
    </dependency>

    <dependency>
        <groupId>com.nimbusds</groupId>
        <artifactId>nimbus-jose-jwt</artifactId>
        <version>8.20</version>
    </dependency>

</dependencies>
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-sdk-bom</artifactId>
            <version>1.0.4</version>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-bom</artifactId>
            <version>2020.0.10</version>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-bom</artifactId>
            <version>4.1.67.Final</version>
        </dependency>
    </dependencies>
</dependencyManagement>

I suspect that there could be a version conflict between the dependencies. I am using Spring core 2.4.0

Please help me to solve this issue. I am geeting delayed.

[alzimmermsft]

Thank you for reporting this issue @nesanmano.

How are you including Spring Core 2.4.0 in your project? Is this also a dependencyManagement inclusion or is this as a parent POM?

Also, looking at your dependencyManagement section the dependencies are missing a few configurations needed to be included as a BOM.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-sdk-bom</artifactId>
            <version>1.0.4</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-bom</artifactId>
            <version>2020.0.10</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-bom</artifactId>
            <version>4.1.67.Final</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

With that change they'll now be able to manage the versions of the dependencies they reference and you should be able to remove the version tags for all dependencies you're using included in them, such as azure-core, reactor-core, etc.

[nesanmano]

I am including spring-core 2.4.0 as part of the parent pom. Regarding the dependency management section, I have taken the important things from the azure-sdk-bom. I am really not sure what is messing everything up.

[alzimmermsft]

Any chance you could send an mvn dependency:tree output of your project? I've attempted to reproduce it based on the dependencies listed above and I didn't run into the Could not initialize class reactor.netty.http.client.HttpClientSecure exception. Additionally, given what HttpClientSecure is is there a specific OS you're using? There may be hooks into OS level SSL handling that may be going awry.

[nesanmano]

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

4.0.0

<groupId>com.sendnconnect</groupId>
<artifactId>cron</artifactId>
<version>1.0.0</version>
<packaging>jar</packaging>

<name>juju</name>
<url>http://maven.apache.org</url>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <jbcrypt.version>0.4</jbcrypt.version>
    <java.version>1.8</java.version>
</properties>

<profiles>
    <profile>
        <id>dev</id>
        <properties>
            <activatedProperties>dev</activatedProperties>
        </properties>
    </profile>
    <profile>
        <id>prod</id>
        <properties>
            <activatedProperties>prod</activatedProperties>
        </properties>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>

        <repositories>
            <repository>
                <id>azure-sdk</id>
                <url>https://github.com/azure/azure-sdk-for-java</url>
            </repository>
        </repositories>
    </profile>
</profiles>

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.4.0</version>
</parent>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
        <version>2.4.0</version>
    </dependency>

    <dependency>
        <groupId>com.twilio.sdk</groupId>
        <artifactId>twilio</artifactId>
        <version>7.47.3</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-jdbc</artifactId>
        <version>2.4.0</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-freemarker</artifactId>
        <version>2.2.4.RELEASE</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-configuration-processor</artifactId>
        <optional>true</optional>
        <version>2.2.4.RELEASE</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-mail</artifactId>
        <version>2.2.4.RELEASE</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-autoconfigure</artifactId>
        <version>2.4.0</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-security</artifactId>
        <version>2.2.4.RELEASE</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-test -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <version>2.4.0</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-lang3</artifactId>
        <version>3.8.1</version>
    </dependency>

    <dependency>
        <groupId>commons-net</groupId>
        <artifactId>commons-net</artifactId>
        <version>3.6</version>
    </dependency>

    <dependency>
        <groupId>com.auth0</groupId>
        <artifactId>java-jwt</artifactId>
        <version>3.4.1</version>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>3.8.1</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>com.drewnoakes</groupId>
        <artifactId>metadata-extractor</artifactId>
        <version>2.6.2</version>
    </dependency>

    <!-- MySql-Connector -->
    <dependency>
        <groupId>mysql</groupId>
        <artifactId>mysql-connector-java</artifactId>
        <version>8.0.13</version>
    </dependency>

    <dependency>
        <groupId>commons-io</groupId>
        <artifactId>commons-io</artifactId>
        <version>2.6</version>
    </dependency>

    <dependency>
        <groupId>net.sourceforge.jexcelapi</groupId>
        <artifactId>jxl</artifactId>
        <version>2.6.12</version>
    </dependency>

    <dependency>
        <groupId>org.apache.xmlbeans</groupId>
        <artifactId>xmlbeans</artifactId>
        <version>2.6.0</version>
    </dependency>

    <dependency>
        <groupId>com.flextrade.jfixture</groupId>
        <artifactId>jfixture</artifactId>
        <version>2.7.2</version>
    </dependency>

    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.18.20</version>
    </dependency>

    <dependency>
        <groupId>net.sf.saxon</groupId>
        <artifactId>Saxon-HE</artifactId>
        <version>9.8.0-7</version>
    </dependency>

    <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
        <version>2.8.5</version>
    </dependency>

    <dependency>
        <groupId>org.im4java</groupId>
        <artifactId>im4java</artifactId>
        <version>1.4.0</version>
    </dependency>

    <dependency>
        <groupId>javax.mail</groupId>
        <artifactId>mail</artifactId>
        <version>1.4</version>
    </dependency>

    <dependency>
        <groupId>com.vimeo.networking</groupId>
        <artifactId>vimeo-networking</artifactId>
        <version>1.1.3</version>
    </dependency>

    <dependency>
        <groupId>org.freemarker</groupId>
        <artifactId>freemarker</artifactId>
        <version>2.3.29</version>
    </dependency>

    <dependency>
        <groupId>org.bouncycastle</groupId>
        <artifactId>bcprov-jdk15on</artifactId>
        <version>1.68</version>
    </dependency>

    <dependency>
        <groupId>org.apache.httpcomponents</groupId>
        <artifactId>httpclient</artifactId>
        <version>4.5.10</version>
    </dependency>

    <dependency>
        <groupId>org.testng</groupId>
        <artifactId>testng</artifactId>
        <version>6.13.1</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-core</artifactId>
        <version>1.2.3</version>
    </dependency>

    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.3</version>
    </dependency>

    <dependency>
        <groupId>org.apache.httpcomponents.core5</groupId>
        <artifactId>httpcore5</artifactId>
        <version>5.1</version>
    </dependency>

    <dependency>
        <groupId>com.doctusoft</groupId>
        <artifactId>json-schema-java7</artifactId>
        <version>1.4.1</version>
    </dependency>

    <dependency>
        <groupId>nu.validator</groupId>
        <artifactId>validator</artifactId>
        <version>15.3.14</version>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <groupId>org.eclipse.jetty</groupId>
                <artifactId>*</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.jetbrains</groupId>
        <artifactId>annotations</artifactId>
        <version>18.0.0</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>javax.inject</groupId>
        <artifactId>javax.inject</artifactId>
        <version>1</version>
    </dependency>

    <dependency>
        <groupId>org.quartz-scheduler</groupId>
        <artifactId>quartz</artifactId>
        <version>2.3.2</version>
    </dependency>

    <dependency>
        <groupId>com.clickntap</groupId>
        <artifactId>vimeo</artifactId>
        <version>1.13</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-administration</artifactId>
        <version>4.0.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-certificates</artifactId>
        <version>4.2.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-jca</artifactId>
        <version>2.0.0</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-keys</artifactId>
        <version>4.3.3</version>
    </dependency>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-secrets</artifactId>
        <version>4.3.3</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core-management</artifactId>
        <version>1.4.1</version>
    </dependency>

    <dependency>
        <groupId>com.azure.resourcemanager</groupId>
        <artifactId>azure-resourcemanager-resources</artifactId>
        <version>2.8.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure.resourcemanager</groupId>
        <artifactId>azure-resourcemanager-keyvault</artifactId>
        <version>2.8.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-keys</artifactId>
        <version>4.3.2</version>
    </dependency>

    <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>msal4j</artifactId>
        <version>1.11.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-administration</artifactId>
        <version>4.0.3</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.3.6</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-tcnative-boringssl-static</artifactId>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core</artifactId>
        <version>1.20.0</version>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core-http-netty</artifactId>
        <version>1.11.0</version>
    </dependency>

    <!-- Reactor -->
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-core</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-test</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor</groupId>
        <artifactId>reactor-tools</artifactId>
        <version>3.4.9</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-extra</artifactId>
        <version>3.4.4</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-adapter</artifactId>
        <version>3.4.4</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.netty</groupId>
        <artifactId>reactor-netty</artifactId>
        <version>1.0.10</version>
    </dependency>

    <dependency>
        <groupId>io.projectreactor.addons</groupId>
        <artifactId>reactor-pool</artifactId>
        <version>0.2.6</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.kafka</groupId>
        <artifactId>reactor-kafka</artifactId>
        <version>1.3.5</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.rabbitmq</groupId>
        <artifactId>reactor-rabbitmq</artifactId>
        <version>1.5.3</version>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.kotlin</groupId>
        <artifactId>reactor-kotlin-extensions</artifactId>
        <version>1.1.4</version>
    </dependency>

    <dependency>
        <groupId>io.netty</groupId>
        <artifactId>netty-handler</artifactId>
        <version>4.1.67.Final</version>
    </dependency>

    <dependency>
        <groupId>com.nimbusds</groupId>
        <artifactId>nimbus-jose-jwt</artifactId>
        <version>8.20</version>
    </dependency>

</dependencies>
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-sdk-bom</artifactId>
            <version>1.0.4</version>
        </dependency>
        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-bom</artifactId>
            <version>2020.0.10</version>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-bom</artifactId>
            <version>4.1.67.Final</version>
        </dependency>
    </dependencies>
</dependencyManagement>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>3.2.0</version>
            <configuration>
                <excludes>
                    <exclude>.git/</exclude>
                    <exclude>.idea/</exclude>
                    <exclude>.settings/</exclude>
                    <exclude>*.gitignore</exclude>
                    <exclude>*.iml</exclude>
                    <exclude>*.project</exclude>
                    <exclude>out/</exclude>
                    <exclude>projects/</exclude>
                    <exclude>*.sql</exclude>
                    <exclude>src/test/*</exclude>
                </excludes>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
            <version>2.2.4.RELEASE</version>
            <configuration>
                <layout>JAR </layout>
                <mainClass>com.sendnconnect.cron.App</mainClass>
            </configuration>
            <executions>
                <execution>
                    <goals>
                        <goal>repackage</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

This is my unit test that causes the error:

import com.azure.core.credential.TokenCredential;
import com.azure.resourcemanager.keyvault.models.Vault;
import com.nomadlogic.juju.azure.AzureKeyVaultAuthenticator;
import org.testng.annotations.Test;
public class AzureTest {

    @Test(priority = 1)
    public void testRespondentsAreReceived() {

        AzureKeyVaultAuthenticator azureKeyVaultAuthenticator = new AzureKeyVaultAuthenticator();

        String clientID = "361eef4e-a589-4c33-9fec-010318274b39";
        String tenantId = "0aa82864-d16f-46ac-991b-41ee90800a10";
        String pfxPassword = "Samsung#2010";
        String pfxPath = "C:\\Users\\Nesan Mano\\Desktop\\sendnconnect.pfx";
        TokenCredential tokenCredential = azureKeyVaultAuthenticator.getTokenCredential(clientID,tenantId,pfxPath,pfxPassword);

        String resouceGroupName  = "sendnconnect";
        String vaultUrl = "https://sendnconnect-vault01.vault.azure.net/";
        Vault keyVault =  azureKeyVaultAuthenticator.getVault(tokenCredential,resouceGroupName,vaultUrl);

        keyVault.keys().getByName("sqlUsername");
    }
}

I am working on a Windows 10 pc.

[alzimmermsft]

@nesanmano, thank you for this!

After digging through this it is an issue with the version of netty-tcnative-boringssl-static being resolved. This is stemming from the parent POM spring-boot-starter-parent:2.4.0, this has a dependency on netty-tcnative-boringssl-static:2.0.34.Final but you're including and depending on a version of reactor-netty which leverages functionality adding in netty-tcnative-boringssl-static:2.0.40.Final.

This is the class being depended on https://github.com/netty/netty-tcnative/blob/main/openssl-dynamic/src/main/java/io/netty/internal/tcnative/AsyncSSLPrivateKeyMethod.java.

Including a direct dependency of netty-tcnative-boringssl-static:2.0.40.Final, or later, or upgrading the version of spring-boot-starter-parent:2.4.8 or later should resolve this issue.

[ghost]

Hi @nesanmano. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[nesanmano]

Hi @alzimmermsft . I have tried to use this netty-tcnative-boringssl-static:2.0.40.Final directly. Unfortunately, I am not getting any success.

[alzimmermsft]

Thanks for the update @nesanmano, is it still throwing the same NoClassDefFoundError when

<dependency>
  <groupId>io.netty</groupId>
  <artifactId>netty-tcnative-boringssl-static</artifactId>
  <version>2.0.40.Final</version>
</dependency>

is being included as the dependency?

[nesanmano]

yes, that is correct. I am trying to upgrade my spring boot to 2.5.5. I will keep you updated.

[alzimmermsft]

@nesanmano, I've realized that upgrading netty-tcnative-boringssl-static only resolved one of the few NoClassDefFoundError in the sample application you gave. I've troubleshot further and found that Reactor Netty depends on functionality in newer versions of netty-common and netty-transport than what spring-boot-starter-parent:2.4.0 was providing to your application. So, I believe the direction you're going with upgrading to Spring Boot 2.5.5 is the optimal solution, you may also be able to upgrade to the latest 2.4.x version (2.4.11) to resolve the issue as well.

[nesanmano]

@alzimmermsft , It works now with 2.5.5.

I am now getting this:

java.lang.IllegalStateException: Please create a subscription before you start resource management. To learn more, see: https://azure.microsoft.com/en-us/free/.

I have an Azure account

[alzimmermsft]

@weidongxu-microsoft do you know the cases that trigger the exception to be thrown?

[weidongxu-microsoft]

@nesanmano

The error message is caused by SDK cannot find any subscription under your account. https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/resourcemanager/azure-resourcemanager-resources/src/main/java/com/azure/resourcemanager/resources/fluentcore/utils/ResourceManagerUtils.java#L123-L126

However it is advised that you always provide a subscription ID to SDK.

If you already have the subscription, would you make it into the AZURE_SUBSCRIPTION_ID env variable, or put it directly into your AzureProfile via ctor new AzureProfile("<YOUR_TENANT_ID>", "<YOUR_SUBSCRIPTION_ID>", AzureEnvironment.AZURE)? You can use null in tenant ID, if you only use limited features that does not depends on tenant.

[nesanmano]

@weidongxu-microsoft Hi, I still keep getting the error message. I am trying it with my trial subscription.

The client 'erwerwerew-3fe9-4990-a8f1-sdfsdfs' with object id 'dsfsfdsf-3fe9-2345-a8f1-sdfsfdsfs' does not have authorization to perform action 'Microsoft.KeyVault/vaults/read' over scope '/subscriptions/dsfsdfdsfsdf-0681-448b-962d-sdfdsfdsfsdfs/resourceGroups/sendnconnect/providers/Microsoft.KeyVault' or the scope is invalid. If access was recently granted, please refresh your credentials.

This is my code:

    AzureProfile azureProfile = new AzureProfile(tenantId,SubscriptionId, AzureEnvironment.AZURE);
    TokenCredential tokenCredential = azureKeyVaultAuthenticator.getTokenCredential(clientID,tenantId,pfxPath,pfxPassword);

    String resouceGroupName  = "sendnconnect";
    String vaultUrl = "https://sendnconnect-vault01.vault.azure.net/";
    Vault keyVault =  azureKeyVaultAuthenticator.getVault(tokenCredential,resouceGroupName,vaultUrl,profile);

    keyVault.keys().getByName("sqlUsername");

[weidongxu-microsoft]

@nesanmano What is this azureKeyVaultAuthenticator?

Please see https://aka.ms/azsdk/java/mgmt for authentication and usage of the SDK.

[nesanmano]

@weidongxu-microsoft

Hi, the azureKeyVaultAuthenticator is my custom class that allows to connect and retrieve keys from the vault.

Why is Azure so hard and complicated?

I have created an application in the AD.

I have added my app to IAM in subscription and added my app to IAM in Key Vault.

I am not sure if the permission is right. It has to be the Key Vault.

This is very weird and fishy

Right now, I am getting this message:

com.azure.core.exception.ResourceModifiedException: Status code 403, "{"error":{"code":"Forbidden","message":"The policy requires the caller 'appid="";oid="";iss=https://sts.windows.net/sdsdwedw-dfdf-46ac-991b-34234k;l23k4;k/' to use on-behalf-of (OBO) flow. For more information on OBO, please see https://go.microsoft.com/fwlink/?linkid=2152310","innererror":{"code":"ForbiddenByPolicy"}}}"

This is my class:

public class AzureKeyVaultAuthenticator {

/**
 * Do certificate based authentication using your PFX file.
 *
 * @param clientId
 *            Also known as applicationId which is received as a part of the app creation process.
 * @param tenantId
 *            Also known as directoryId which is received as a part of the app creation process.
 * @param pathPfx
 *            Path to your PFX certificate.
 * @param pfxPassword
 *            Password to your PFX certificate, this can be empty if that's the value given when it was created.
 */
public TokenCredential getTokenCredential(String clientId, String tenantId, String pathPfx, String pfxPassword) {

    TokenCredential credential = new ClientCertificateCredentialBuilder()
            .clientId(clientId)
            .tenantId(tenantId)
            .pfxCertificate(pathPfx, pfxPassword)
            .build();

    return credential;
}

/**
 * Find the vault you want to operate on by keyVaultName.
 *
 * @param credential
 *            Credential to authenticate a {@link KeyVaultManager} with.
 * @param resourceGroupName
 *            The name of the resource group your Key Vault is a part of.
 * @param vaultBaseUrl
 *            The URL that identifies your Key Vault.
 * @return A {@link Vault} object representing your Key Vault.
 */
public Vault getVault(TokenCredential credential, String resourceGroupName, String vaultBaseUrl,AzureProfile profile) {

    KeyVaultManager manager = KeyVaultManager.authenticate(credential, profile);

    Optional<Vault> optional = manager
            .vaults()
            .listByResourceGroup(resourceGroupName)
            .stream()
            .filter(vault -> vaultBaseUrl.equals(vault.vaultUri()))
            .findFirst();
    if (optional.isPresent()) {
        return optional.get();
    }
    return null;
}}

[weidongxu-microsoft]

@nesanmano

Keyvault is a bit hard to understand, and it got more configure on permission (well, because it is key vault).

When you create (or modify) it, you can see it has 2 approach to permission.

One is the access policy defined within Keyvault, and if you use this, IAM does not apply, you need to add your appId here.

Another (newer approach) is the RBAC. If you use this, you can use IAM with e.g. KEY_VAULT_CRYPTO_USER, KEY_VAULT_SECRETS_USER

[weidongxu-microsoft]

Also let me know which call result in 403. I don't think listByResourceGroup will get this (above permission is about getting key/secret from key vault, list does not require special handline other than IAM)?

[nesanmano]

@weidongxu-microsoft Hi, sorry for the delay, I sent some time on reading and I have now set and configured everything, I am able to question the key vault using my Java app. I am looking for a best practice to retrieve those keys and store them in environment variables in my Linux container.

[nesanmano]

I have two proposals.

1. I am looking for a way to fetch keys from the vault and store them in environment variables on my linux server.
2. Fetch them every time the app(Java) needs it.

Which one is same.?

[weidongxu-microsoft]

@nesanmano I assume you are using one of these libs for retrieving the key/secret. https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/keyvault

@alzimmermsft Hi Alan, do you have recommendation for the question above? I am not very familiar with best practice on keyvault. And since it is about security, expert opinion is preferred :-)

[vcolin7]

Hi @nesanmano, in regards to the keys question, Key Vault is a service that has low limits on how many calls you can make per second, so if you would need to use the a key constantly I would prefer to cache it locally. In the long term, since the key is likely to expire at some point, I would have some logic in place to refresh its value whenever this happens.

I also noticed you are using a third party library to connect retrieve your keys from Key Vault, have you considered switching to our Azure Security Key Vault Keys client library? We constantly add features, improvements and bug fixes and have plenty of documentation and samples.

[nesanmano]

@weidongxu-microsoft Yes, I am using that library. In order to make things better, I am looking to run a shell script and store those key as env var for that local user(my app)

@vcolin7 I am using azure library. I am precisely using the Vault object from azure sdk. Below is my Java code for my unit test

 AzureProfile azureProfile = new AzureProfile(tenantId,SubscriptionId, AzureEnvironment.AZURE);
    TokenCredential tokenCredential = azureKeyVaultAuthenticator.getTokenCredential(clientID,tenantId,pfxPath,pfxPassword);

    String resouceGroupName  = "xxxxxxxxxxxxx";
    String vaultUrl = "https://xxxxxxxxxxxxxx-vault01.vault.azure.net/";
    Vault keyVault =  azureKeyVaultAuthenticator.getVault(tokenCredential,resouceGroupName,vaultUrl,azureProfile);

    keyVault.keys().getByName("sqlUsername");

azureKeyVaultAuthenticator (this is a custom class in which I use the sdk.)

---

I have the following script (shell). The issue it is not storing my keys. It is kind of storing it in a temporary child process which I have not idea. I can't even find the key it has stored temporily.

!/usr/bin/env bash

#

Fetch secrets for local development from Azure KeyVault

and print them to stdout as a bunch of env var exports.

These secrets should be added to your local .env file

to enable running integration tests locally.

# KEY_VAULT=$1

function fetch_secret_from_keyvault() { local SECRET_NAME=$1

az keyvault secret show --vault-name "${KEY_VAULT}" --name "${SECRET_NAME}" --query "value"

}

function store_secret_from_keyvault() { local SECRET_VAR=$1 local SECRET_NAME=$2

local SECRET_VALUE=`fetch_secret_from_keyvault "${SECRET_NAME}"`
store_secret "${SECRET_VAR}" "${SECRET_VALUE}"

}

function store_secret() { local SECRET_VAR=$1 local SECRET_VALUE=$2

echo "export sss${SECRET_VAR}=${SECRET_VALUE}"

}

echo "# ----------------------- " echo "# Fetched the following secrets from ${KEY_VAULT} on "date

az login --service-principal --username '361eef4e-1234-rrrrr-9fec-serewsrwrwerwer' --tenant '0dsfsdfsdfsda4-34543-4535-3454-asdasdwesdc' --password './xxxxxxxxxxx.pem'

store_secret_from_keyvault "MONGO_URI" "MONGO-URI" store_secret_from_keyvault "WASB_MEDIA_STORAGE_ACCOUNT_NAME" "local-dev-media-storage-account-name" store_secret_from_keyvault "WASB_MEDIA_STORAGE_ACCOUNT_KEY" "local-dev-media-storage-account-key" store_secret_from_keyvault "WASB_MEDIA_STORAGE_CONTAINER_NAME" "local-dev-media-storage-container-name"

store_secret "KEY_VAULT_URI" "https://${KEY_VAULT}.vault.azure.net/" store_secret_from_keyvault "KEY_VAULT_CLIENT_ID" "kv-sp-app-id" store_secret_from_keyvault "KEY_VAULT_CLIENT_SECRET" "kv-sp-password" store_secret_from_keyvault "KEY_VAULT_TENANT_ID" "kv-sp-tenant"

echo "# End of fetched secrets. " echo "# ----------------------- "

[vcolin7]

Oh sorry @nesanmano, I misread the conversation above and didn't notice AzureKeyVaultAuthenticator is your own custom class. Based on the code I saw there, I see you are using our resource management library for Key Vault, which allows for you to create and manage vaults, perform role-based access control, and do some simple operations like creating and fetching secrets/keys/certificates. If you want to manipulate the resources in your key vault and don't really care about managing the key vault itself, I would recommend you use the libraries I mentioned in my previous comment.

That said, I am a bit confused as to what you are trying to accomplish, so let me see if I got things straight: the shell script you just shared locally stores some secrets from your key vault as environment variables which then you want your Java application to access, correct? As far as I know, using export in a process saves the variables only in the memory of the process itself, in other words, the variables are not known outside of the process. You can access those values from outside by following the steps laid out in the StackOverflow answer I shared, but I'm sure there's simpler ways to accomplish what you want to do. You could have your shell script also start your Java application after setting the variables, so the child process where the Java app resides has access to them via System.getenv("<name-of-your-variable>"), for example.

I would go for an alternative route without a shell script and do something like the following to get the Java app to access your secrets:

SecretClient secretClient = new SecretClientBuilder()
    .vaultUrl(vaultUrl)
    .credential(tokenCredential)
    .buildClient();

KeyVaultSecret mySecret = secretClient.getSecret(secretName);

System.out.println("Retrievved secret with name: " + mySecret.getName() + " and value: " + mySecret.getValue());

And then use mySecret wherever I need it in the Java app. If the secret changes in the future, you can use the same SecretClient you already possess to fetch its new value.

Finally, if you decide to keep using the Key Vault Management library and assuming this "sqlUsername" is stored as a secret in your vault, your unit test should use the following code

Secret mySecret = keyVault.secrets().getByName("sqlUsername");

Instead of keyVault.keys().getByName("sqlUsername");, since that looks for a key with that name (keys and secrets are different entities in the context of Azure Key Vault).

[nesanmano]

@vcolin7 Hi, You have totally understood my concern. I am actually using you way to retrieve the key in my unit test. My concern is that I don't want to use it very often. Frequent queries to the vault is not very cost effective from my understanding. Hence, I am looking to run a custom script every time the server boots up and the software would simply get them from the env.

[vcolin7]

Then I think you can do whatever you want: either using Java code or a script to set/get the values you need when booting up the software. There's no reason to make a get call every time you need a secret unless you think their values will change regularly. If they can change during the lifecycle of the application, you might want to have some Java code to refresh them without needing to restart your software. You can use the Key Vault Secrets client library to check the version of a secret via KeyVaultSecret.getProperties().getVersion().

[alzimmermsft]

@nesanmano, would it be fine to close this issue as the original purpose was for the java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientSecure and that has been resolved?

[nesanmano]

@alzimmermsft Hi, yeah I should have closed this but I was having issues related to satellite things that revolves around azure. I had a good time here. I learnt a lot about Azure. Thanks to everyone.