search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.36k stars 2k forks source link

[BUG] Reactor/Netty dependencies issues when using KeyVault and Service Bus libraries #22242

Closed Paizo closed 3 years ago

Paizo commented 3 years ago

Describe the bug

Simple java function that get messages from a service bus queue and fetch a certificate from the keyvault. During the keyvault initialization a NoSuchMethod error occur

Exception or Stack Trace

Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: Failure\nException: NoSuchMethodError: io.netty.handler.logging.LoggingHandler.<init>(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V\nStack: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodExecutorImpl.execute(JavaMethodExecutorImpl.java:54)\n\tat com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:57)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)\n\tat com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)\n\tat com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.NoSuchMethodError: io.netty.handler.logging.LoggingHandler.<init>(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V\n\tat reactor.netty.transport.logging.ReactorNettyLoggingHandler.<init>(ReactorNettyLoggingHandler.java:63)\n\tat reactor.netty.transport.logging.AdvancedByteBufFormat.toLoggingHandler(AdvancedByteBufFormat.java:109)\n\tat reactor.netty.http.client.HttpClientConfig.<clinit>(HttpClientConfig.java:635)\n\tat reactor.netty.http.client.HttpClientConnect.<init>(HttpClientConnect.java:84)\n\tat reactor.netty.http.client.HttpClient.create(HttpClient.java:391)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17)\n\tat com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:50)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:40)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildAsyncClient(CertificateClientBuilder.java:167)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildClient(CertificateClientBuilder.java:109)

To Reproduce

create a skeleton java function application using a @ServiceBusQueueTrigger create a keyvault with the function list/get permissions try to fetch a certificate from the keyvault from within the function

Code Snippet

        certificateClient = new CertificateClientBuilder()
                .vaultUrl(keyVaultUrl)
                .credential(new ManagedIdentityCredentialBuilder().build())
                .buildClient();
        this.secretClient = new SecretClientBuilder()
                .vaultUrl(keyVaultUrl)
                .credential(new ManagedIdentityCredentialBuilder().build())
                .buildClient();

Expected behavior

No exceptions

Screenshots If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

com.microsoft.azure.functions:azure-functions-java-library:1.4.2
com.microsoft.azure:azure-servicebus:3.6.4
com.azure:azure-security-keyvault-secrets:4.2.8
com.azure:azure-security-keyvault-certificates:4.1.8
com.azure:azure-identity:1.3.1

maven dependencies tree:

[INFO] +- com.microsoft.azure.functions:azure-functions-java-library:jar:1.4.2:compile [INFO] +- com.microsoft.azure:azure-servicebus:jar:3.6.4:compile [INFO] | +- org.apache.qpid:proton-j:jar:0.33.4:compile [INFO] | +- com.microsoft.azure:qpid-proton-j-extensions:jar:1.2.3:compile [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile [INFO] | +- org.asynchttpclient:async-http-client:jar:2.12.1:compile [INFO] | | +- org.asynchttpclient:async-http-client-netty-utils:jar:2.12.1:compile [INFO] | | +- io.netty:netty-codec-socks:jar:4.1.48.Final:compile [INFO] | | +- io.netty:netty-handler-proxy:jar:4.1.48.Final:compile [INFO] | | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.48.Final:compile [INFO] | | +- io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.48.Final:compile [INFO] | | +- org.reactivestreams:reactive-streams:jar:1.0.3:compile [INFO] | | +- com.typesafe.netty:netty-reactive-streams:jar:2.0.4:compile [INFO] | | - com.sun.activation:javax.activation:jar:1.2.0:compile [INFO] | - com.microsoft.azure:azure-client-authentication:jar:1.7.3:compile [INFO] | +- com.microsoft.azure:azure-client-runtime:jar:1.7.3:compile [INFO] | | - com.microsoft.rest:client-runtime:jar:1.7.3:compile [INFO] | | +- com.squareup.retrofit2:retrofit:jar:2.7.2:compile [INFO] | | +- com.squareup.okhttp3:okhttp:jar:3.12.6:compile [INFO] | | | - com.squareup.okio:okio:jar:1.15.0:compile [INFO] | | +- com.squareup.okhttp3:logging-interceptor:jar:3.12.2:compile [INFO] | | +- com.squareup.okhttp3:okhttp-urlconnection:jar:3.12.2:compile [INFO] | | +- com.squareup.retrofit2:converter-jackson:jar:2.7.2:compile [INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-joda:jar:2.10.1:compile [INFO] | | +- io.reactivex:rxjava:jar:1.3.8:compile [INFO] | | - com.squareup.retrofit2:adapter-rxjava:jar:2.7.2:compile [INFO] | +- com.microsoft.azure:adal4j:jar:1.6.4:compile [INFO] | | - com.google.code.gson:gson:jar:2.8.0:compile [INFO] | - com.microsoft.azure:azure-annotations:jar:1.10.0:compile [INFO] +- org.apache.httpcomponents.client5:httpclient5:jar:5.0.3:compile [INFO] | +- org.apache.httpcomponents.core5:httpcore5:jar:5.0.2:compile [INFO] | +- org.apache.httpcomponents.core5:httpcore5-h2:jar:5.0.2:compile [INFO] | - commons-codec:commons-codec:jar:1.13:compile [INFO] +- com.azure:azure-security-keyvault-secrets:jar:4.2.8:compile [INFO] | +- com.azure:azure-core:jar:1.16.0:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.2:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-core:jar:2.12.2:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.2:compile [INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.2:compile [INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.12.2:compile [INFO] | | | +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.12.2:compile [INFO] | | | | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:compile [INFO] | | | | - jakarta.activation:jakarta.activation-api:jar:1.2.1:compile [INFO] | | | +- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile [INFO] | | | - com.fasterxml.woodstox:woodstox-core:jar:6.2.4:compile [INFO] | | +- io.projectreactor:reactor-core:jar:3.4.5:compile [INFO] | | - io.netty:netty-tcnative-boringssl-static:jar:2.0.38.Final:compile [INFO] | - com.azure:azure-core-http-netty:jar:1.9.2:compile [INFO] | +- io.netty:netty-codec-http2:jar:4.1.63.Final:compile [INFO] | +- io.netty:netty-transport-native-unix-common:jar:4.1.63.Final:compile [INFO] | - io.projectreactor.netty:reactor-netty:jar:1.0.6:compile [INFO] | +- io.projectreactor.netty:reactor-netty-core:jar:1.0.6:compile [INFO] | | +- io.netty:netty-resolver-dns:jar:4.1.63.Final:compile [INFO] | | | - io.netty:netty-codec-dns:jar:4.1.63.Final:compile [INFO] | | - io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.63.Final:compile [INFO] | +- io.projectreactor.netty:reactor-netty-http:jar:1.0.6:compile [INFO] | - io.projectreactor.netty:reactor-netty-http-brave:jar:1.0.6:runtime [INFO] | - io.zipkin.brave:brave-instrumentation-http:jar:5.13.3:runtime [INFO] | - io.zipkin.brave:brave:jar:5.13.3:runtime [INFO] | - io.zipkin.reporter2:zipkin-reporter-brave:jar:2.16.3:runtime [INFO] | - io.zipkin.reporter2:zipkin-reporter:jar:2.16.3:runtime [INFO] | - io.zipkin.zipkin2:zipkin:jar:2.23.2:runtime [INFO] +- com.azure:azure-security-keyvault-certificates:jar:4.1.8:compile [INFO] +- com.azure:azure-identity:jar:1.3.1:compile [INFO] | +- com.microsoft.azure:msal4j:jar:1.10.0:compile [INFO] | | - com.nimbusds:oauth2-oidc-sdk:jar:9.4:compile [INFO] | | +- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile [INFO] | | +- com.nimbusds:content-type:jar:2.1:compile [INFO] | | +- net.minidev:json-smart:jar:2.4.2:compile (version selected from constraint [1.3.2,2.4.2]) [INFO] | | | - net.minidev:accessors-smart:jar:2.4.2:compile [INFO] | | | - org.ow2.asm:asm:jar:8.0.1:compile [INFO] | | +- com.nimbusds:lang-tag:jar:1.5:compile [INFO] | | - com.nimbusds:nimbus-jose-jwt:jar:9.8.1:compile [INFO] | +- com.microsoft.azure:msal4j-persistence-extension:jar:1.1.0:compile [INFO] | | - net.java.dev.jna:jna:jar:5.5.0:compile [INFO] | +- net.java.dev.jna:jna-platform:jar:5.6.0:compile [INFO] | - org.linguafranca.pwdb:KeePassJava2:jar:2.1.4:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-kdb:jar:2.1.4:compile [INFO] | | +- org.linguafranca.pwdb:database:jar:2.1.4:compile [INFO] | | | - com.madgag.spongycastle:core:jar:1.54.0.0:compile [INFO] | | - org.jetbrains:annotations:jar:15.0:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-dom:jar:2.1.4:compile [INFO] | | - org.linguafranca.pwdb:KeePassJava2-kdbx:jar:2.1.4:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-jaxb:jar:2.1.4:compile [INFO] | - org.linguafranca.pwdb:KeePassJava2-simple:jar:2.1.4:compile [INFO] | +- org.simpleframework:simple-xml:jar:2.7.1:compile [INFO] | | +- stax:stax-api:jar:1.0.1:compile [INFO] | | - stax:stax:jar:1.2.0:compile [INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.5:compile [INFO] | - com.fasterxml:aalto-xml:jar:1.0.0:compile [INFO] +- xpp3:xpp3:jar:1.1.4c:compile [INFO] +- org.slf4j:log4j-over-slf4j:jar:1.7.30:compile [INFO] +- ch.qos.logback:logback-classic:jar:1.2.3:compile [INFO] | - ch.qos.logback:logback-core:jar:1.2.3:compile [INFO] +- org.projectlombok:lombok:jar:1.18.20:provided [INFO] +- org.junit.jupiter:junit-jupiter:jar:5.7.1:test [INFO] | +- org.junit.jupiter:junit-jupiter-api:jar:5.7.1:test [INFO] | | +- org.apiguardian:apiguardian-api:jar:1.1.0:test [INFO] | | +- org.opentest4j:opentest4j:jar:1.2.0:test [INFO] | | - org.junit.platform:junit-platform-commons:jar:1.7.1:test [INFO] | +- org.junit.jupiter:junit-jupiter-params:jar:5.7.1:test [INFO] | - org.junit.jupiter:junit-jupiter-engine:jar:5.7.1:test [INFO] | - org.junit.platform:junit-platform-engine:jar:1.7.1:test [INFO] +- org.mockito:mockito-core:jar:3.9.0:test [INFO] | +- net.bytebuddy:byte-buddy:jar:1.10.20:test [INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.10.20:test [INFO] | - org.objenesis:objenesis:jar:3.2:test [INFO] +- org.mockito:mockito-junit-jupiter:jar:3.9.0:test [INFO] +- org.mock-server:mockserver-netty:jar:5.11.2:test [INFO] | +- org.mock-server:mockserver-core:jar:5.11.2:test [INFO] | | +- com.lmax:disruptor:jar:3.4.2:test [INFO] | | +- javax.servlet:javax.servlet-api:jar:4.0.1:test [INFO] | | +- com.jcraft:jzlib:jar:1.1.3:test [INFO] | | +- com.fasterxml.uuid:java-uuid-generator:jar:4.0.1:test [INFO] | | +- org.apache.velocity:velocity-engine-scripting:jar:2.2:test [INFO] | | +- org.apache.velocity:velocity-engine-core:jar:2.2:test [INFO] | | +- net.javacrumbs.json-unit:json-unit-core:jar:2.19.0:test [INFO] | | | - org.hamcrest:hamcrest-core:jar:2.2:test [INFO] | | | - org.hamcrest:hamcrest:jar:2.2:test [INFO] | | +- com.github.java-json-tools:json-schema-validator:jar:2.2.14:test [INFO] | | | +- com.github.java-json-tools:jackson-coreutils-equivalence:jar:1.0:test [INFO] | | | | - com.github.java-json-tools:jackson-coreutils:jar:2.0:test [INFO] | | | | - com.github.java-json-tools:msg-simple:jar:1.2:test [INFO] | | | | - com.github.java-json-tools:btf:jar:1.3:test [INFO] | | | +- com.github.java-json-tools:json-schema-core:jar:1.2.14:test [INFO] | | | | +- com.github.java-json-tools:uri-template:jar:0.10:test [INFO] | | | | - org.mozilla:rhino:jar:1.7.7.2:test [INFO] | | | +- com.sun.mail:mailapi:jar:1.6.2:test [INFO] | | | +- joda-time:joda-time:jar:2.10.5:compile [INFO] | | | +- com.googlecode.libphonenumber:libphonenumber:jar:8.11.1:test [INFO] | | | - net.sf.jopt-simple:jopt-simple:jar:5.0.4:test [INFO] | | +- com.jayway.jsonpath:json-path:jar:2.4.0:test [INFO] | | +- io.swagger.parser.v3:swagger-parser:jar:2.0.23:test [INFO] | | | +- io.swagger.parser.v3:swagger-parser-v2-converter:jar:2.0.23:test [INFO] | | | | +- io.swagger:swagger-core:jar:1.6.2:test [INFO] | | | | | +- io.swagger:swagger-models:jar:1.6.2:test [INFO] | | | | | | - io.swagger:swagger-annotations:jar:1.6.2:test [INFO] | | | | | - javax.validation:validation-api:jar:1.1.0.Final:test [INFO] | | | | +- io.swagger:swagger-parser:jar:1.0.52:test [INFO] | | | | | - org.slf4j:slf4j-ext:jar:1.7.28:test [INFO] | | | | +- io.swagger:swagger-compat-spec-parser:jar:1.0.52:test [INFO] | | | | | - org.apache.httpcomponents:httpclient:jar:4.5.2:test [INFO] | | | | | - commons-logging:commons-logging:jar:1.2:test [INFO] | | | | +- io.swagger.core.v3:swagger-models:jar:2.1.5:test [INFO] | | | | - io.swagger.parser.v3:swagger-parser-core:jar:2.0.23:test [INFO] | | | - io.swagger.parser.v3:swagger-parser-v3:jar:2.0.23:test [INFO] | | | +- io.swagger.core.v3:swagger-core:jar:2.1.5:test [INFO] | | | | +- io.swagger.core.v3:swagger-annotations:jar:2.1.5:test [INFO] | | | | - jakarta.validation:jakarta.validation-api:jar:2.0.2:test [INFO] | | | - com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.11.1:test [INFO] | | | - org.yaml:snakeyaml:jar:1.26:test [INFO] | | +- javax.xml.bind:jaxb-api:jar:2.4.0-b180830.0359:test [INFO] | | | - javax.activation:javax.activation-api:jar:1.2.0:test [INFO] | | +- org.xmlunit:xmlunit-core:jar:2.8.0:test [INFO] | | +- org.xmlunit:xmlunit-placeholders:jar:2.8.0:test [INFO] | | - org.apache.commons:commons-text:jar:1.9:test [INFO] | +- io.netty:netty-buffer:jar:4.1.53.Final:compile [INFO] | +- io.netty:netty-codec:jar:4.1.53.Final:compile [INFO] | +- io.netty:netty-codec-http:jar:4.1.53.Final:compile [INFO] | +- io.netty:netty-common:jar:4.1.53.Final:compile [INFO] | +- io.netty:netty-handler:jar:4.1.53.Final:compile [INFO] | | - io.netty:netty-resolver:jar:4.1.53.Final:compile [INFO] | +- io.netty:netty-transport:jar:4.1.53.Final:compile [INFO] | +- commons-io:commons-io:jar:2.8.0:test [INFO] | - com.google.guava:guava:jar:30.0-android:compile [INFO] | +- com.google.guava:failureaccess:jar:1.0.1:compile [INFO] | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile [INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.2:compile [INFO] | +- org.checkerframework:checker-compat-qual:jar:2.5.5:compile [INFO] | +- com.google.errorprone:error_prone_annotations:jar:2.3.4:compile [INFO] | - com.google.j2objc:j2objc-annotations:jar:1.3:compile [INFO] +- org.mock-server:mockserver-client-java:jar:5.11.2:test [INFO] | - org.apache.commons:commons-lang3:jar:3.11:compile [INFO] +- org.mock-server:mockserver-junit-jupiter:jar:5.11.2:test [INFO] +- com.github.stefanbirkner:system-lambda:jar:1.2.0:test [INFO] - uk.org.webcompere:system-stubs-jupiter:jar:1.2.0:test [INFO] - uk.org.webcompere:system-stubs-core:jar:1.2.0:test

Additional context

Deploy the function in a function app service, place a message in the queue, the function get triggered but fails to connect to the keyvault to fetch a certificate.

Tried to mitigate by adding reactor/netty dependencies with no success:

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-all</artifactId>
            <version>4.1.63.Final</version>
        </dependency>

        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty-http</artifactId>
            <version>3.4.5</version>
        </dependency>

result in chasing other exceptions, with no luck so far:

2021-06-10T09:10:09.804094220Z Caused by: java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig
2021-06-10T09:10:09.804097920Z  at reactor.netty.http.client.HttpClientConnect.<init>(HttpClientConnect.java:84)
2021-06-10T09:10:09.804101820Z  at reactor.netty.http.client.HttpClient.create(HttpClient.java:391)
2021-06-10T09:10:09.804352328Z  at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97)
2021-06-10T09:10:09.804373429Z  at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17)
2021-06-10T09:10:09.804377729Z  at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58)
2021-06-10T09:10:09.804381429Z  at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50)
2021-06-10T09:10:09.804385030Z  at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40)
2021-06-10T09:10:09.804388630Z  at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62)
2021-06-10T09:10:09.804392230Z  at com.azure.security.keyvault.certificates.CertificateClientBuilder.buildAsyncClient(CertificateClientBuilder.java:167)
2021-06-10T09:10:09.804395930Z  at com.azure.security.keyvault.certificates.CertificateClientBuilder.buildClient(CertificateClientBuilder.java:109)

Information Checklist

Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

alzimmermsft commented 3 years ago

Thank you for submitting this issue @Paizo, could you try using azure-sdk-bom 1.0.3 (guide on using it) to manage your Azure SDKs?

Looking at the dependencies you're using the Service Bus version being used is resulting in some Netty versions being resolved to a much older version, 4.1.48, compared to what is expected by the KeyVault SDKs, 4.1.63. I recommend using the azure-sdk-bom as it will manage both Azure SDK versions which have dependency alignment and the versions of the dependencies being used by the Azure SDKs.

derekg52 commented 3 years ago

@alzimmermsft I've had a nearly identical situation rattling around for a week or so.

[2021-06-11T19:33:28.193Z] Caused by: java.lang.NoSuchMethodError: io.netty.handler.logging.LoggingHandler.(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V [2021-06-11T19:33:28.193Z] at reactor.netty.transport.logging.ReactorNettyLoggingHandler.(ReactorNettyLoggingHandler.java:63) [2021-06-11T19:33:28.199Z] at reactor.netty.transport.logging.AdvancedByteBufFormat.toLoggingHandler(AdvancedByteBufFormat.java:109) [2021-06-11T19:33:28.200Z] at reactor.netty.http.client.HttpClientConfig.(HttpClientConfig.java:635) [2021-06-11T19:33:28.201Z] at reactor.netty.http.client.HttpClientConnect.(HttpClientConnect.java:84) [2021-06-11T19:33:28.202Z] at reactor.netty.http.client.HttpClient.create(HttpClient.java:391) [2021-06-11T19:33:28.203Z] at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97) [2021-06-11T19:33:28.204Z] at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17) [2021-06-11T19:33:28.205Z] at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58) [2021-06-11T19:33:28.206Z] at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) [2021-06-11T19:33:28.207Z] at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) [2021-06-11T19:33:28.213Z] at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62) [2021-06-11T19:33:28.214Z] at com.azure.data.appconfiguration.ConfigurationClientBuilder.buildAsyncClient(ConfigurationClientBuilder.java:203) [2021-06-11T19:33:28.215Z] at com.azure.data.appconfiguration.ConfigurationClientBuilder.buildClient(ConfigurationClientBuilder.java:124)

I tried your solution of using the 1.03 BOM and the additional recommended dependencies such as Netty but no luck. I tried different variations of what people have mentioned in the past of explicitly including Netty also, cleaned out my .m2 folder locally, etc.

Here's my dependency tree - notice Netty is 4.1.63 here:

[INFO] com.willistowerswatson:BIC_Service:jar:1.0-SNAPSHOT [INFO] +- com.azure:azure-storage-blob:jar:12.11.1:compile [INFO] | +- com.azure:azure-core-http-netty:jar:1.9.2:compile [INFO] | | +- io.netty:netty-handler:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-common:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-resolver:jar:4.1.63.Final:compile [INFO] | | | +- io.netty:netty-transport:jar:4.1.63.Final:compile [INFO] | | | - io.netty:netty-codec:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-handler-proxy:jar:4.1.63.Final:compile [INFO] | | | - io.netty:netty-codec-socks:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-buffer:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-codec-http:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-codec-http2:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-transport-native-unix-common:jar:4.1.63.Final:compile [INFO] | | +- io.netty:netty-transport-native-epoll:jar:linux-x86_64:4.1.63.Final:compile [INFO] | | - io.netty:netty-transport-native-kqueue:jar:osx-x86_64:4.1.63.Final:compile [INFO] | +- com.azure:azure-storage-common:jar:12.11.1:compile [INFO] | - com.azure:azure-storage-internal-avro:jar:12.0.4:compile [INFO] +- com.azure:azure-cosmos:jar:4.15.0:compile [INFO] | +- com.fasterxml.jackson.module:jackson-module-afterburner:jar:2.12.2:compile [INFO] | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.2:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.12.2:compile [INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.12.2:compile [INFO] | +- org.hdrhistogram:HdrHistogram:jar:2.1.12:compile [INFO] | +- io.dropwizard.metrics:metrics-core:jar:4.1.19:compile [INFO] | - io.micrometer:micrometer-core:jar:1.6.6:compile [INFO] | - org.latencyutils:LatencyUtils:jar:2.0.3:runtime [INFO] +- com.azure:azure-identity:jar:1.3.0:compile [INFO] | +- com.microsoft.azure:msal4j:jar:1.10.0:compile [INFO] | | - com.nimbusds:oauth2-oidc-sdk:jar:9.4:compile [INFO] | | +- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:compile [INFO] | | +- com.nimbusds:content-type:jar:2.1:compile [INFO] | | +- net.minidev:json-smart:jar:2.4.2:compile (version selected from constraint [1.3.2,2.4.2]) [INFO] | | | - net.minidev:accessors-smart:jar:2.4.2:compile [INFO] | | | - org.ow2.asm:asm:jar:8.0.1:compile [INFO] | | +- com.nimbusds:lang-tag:jar:1.5:compile [INFO] | | - com.nimbusds:nimbus-jose-jwt:jar:9.8.1:compile [INFO] | +- com.microsoft.azure:msal4j-persistence-extension:jar:1.1.0:compile [INFO] | | - net.java.dev.jna:jna:jar:5.5.0:compile [INFO] | +- net.java.dev.jna:jna-platform:jar:5.6.0:compile [INFO] | - org.linguafranca.pwdb:KeePassJava2:jar:2.1.4:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-kdb:jar:2.1.4:compile [INFO] | | +- org.linguafranca.pwdb:database:jar:2.1.4:compile [INFO] | | | - com.madgag.spongycastle:core:jar:1.54.0.0:compile [INFO] | | - org.jetbrains:annotations:jar:15.0:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-dom:jar:2.1.4:compile [INFO] | | - org.linguafranca.pwdb:KeePassJava2-kdbx:jar:2.1.4:compile [INFO] | +- org.linguafranca.pwdb:KeePassJava2-jaxb:jar:2.1.4:compile [INFO] | - org.linguafranca.pwdb:KeePassJava2-simple:jar:2.1.4:compile [INFO] | +- org.simpleframework:simple-xml:jar:2.7.1:compile [INFO] | | +- stax:stax-api:jar:1.0.1:compile [INFO] | | +- stax:stax:jar:1.2.0:compile [INFO] | | - xpp3:xpp3:jar:1.1.3.3:compile [INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.5:compile [INFO] | - com.fasterxml:aalto-xml:jar:1.0.0:compile [INFO] +- com.azure:azure-security-keyvault-secrets:jar:4.2.8:compile [INFO] +- com.azure:azure-data-appconfiguration:jar:1.1.12:compile [INFO] +- com.azure:azure-core:jar:1.16.0:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.12.2:compile [INFO] | | +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.12.2:compile [INFO] | | | +- jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.2:compile [INFO] | | | - jakarta.activation:jakarta.activation-api:jar:1.2.1:compile [INFO] | | +- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile [INFO] | | - com.fasterxml.woodstox:woodstox-core:jar:6.2.4:compile [INFO] | +- io.projectreactor:reactor-core:jar:3.4.5:compile [INFO] | | - org.reactivestreams:reactive-streams:jar:1.0.3:compile [INFO] | - io.netty:netty-tcnative-boringssl-static:jar:2.0.38.Final:compile [INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.2:compile [INFO] +- io.projectreactor.netty:reactor-netty:jar:1.0.6:compile [INFO] | +- io.projectreactor.netty:reactor-netty-core:jar:1.0.6:compile [INFO] | | +- io.netty:netty-resolver-dns:jar:4.1.63.Final:compile [INFO] | | | - io.netty:netty-codec-dns:jar:4.1.63.Final:compile [INFO] | | - io.netty:netty-resolver-dns-native-macos:jar:osx-x86_64:4.1.63.Final:compile [INFO] | +- io.projectreactor.netty:reactor-netty-http:jar:1.0.6:compile [INFO] | - io.projectreactor.netty:reactor-netty-http-brave:jar:1.0.6:runtime [INFO] | - io.zipkin.brave:brave-instrumentation-http:jar:5.13.3:runtime [INFO] | - io.zipkin.brave:brave:jar:5.13.3:runtime [INFO] | - io.zipkin.reporter2:zipkin-reporter-brave:jar:2.16.3:runtime [INFO] | - io.zipkin.reporter2:zipkin-reporter:jar:2.16.3:runtime [INFO] | - io.zipkin.zipkin2:zipkin:jar:2.23.2:runtime [INFO] +- org.slf4j:slf4j-api:jar:1.7.30:compile [INFO] +- com.auth0:jwks-rsa:jar:0.11.0:compile [INFO] | +- commons-codec:commons-codec:jar:1.13:compile [INFO] | +- commons-io:commons-io:jar:2.6:runtime [INFO] | - com.google.guava:guava:jar:27.1-jre:compile [INFO] | +- com.google.guava:failureaccess:jar:1.0.1:compile [INFO] | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile [INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.2:compile [INFO] | +- org.checkerframework:checker-qual:jar:2.5.2:compile [INFO] | +- com.google.errorprone:error_prone_annotations:jar:2.2.0:compile [INFO] | +- com.google.j2objc:j2objc-annotations:jar:1.1:compile [INFO] | - org.codehaus.mojo:animal-sniffer-annotations:jar:1.17:compile [INFO] +- com.microsoft.azure.functions:azure-functions-java-library:jar:1.4.0:compile [INFO] +- javax.servlet:javax.servlet-api:jar:4.0.1:provided [INFO] +- com.microsoft.graph:microsoft-graph:jar:1.7.1:compile [INFO] | +- com.sun.jersey:jersey-server:jar:1.19.4:compile [INFO] | | - com.sun.jersey:jersey-core:jar:1.19.4:compile [INFO] | | - javax.ws.rs:jsr311-api:jar:1.1.1:compile [INFO] | +- com.microsoft.graph:microsoft-graph-core:jar:1.0.0:compile [INFO] | | +- com.googlecode.json-simple:json-simple:jar:1.1:compile [INFO] | | - com.squareup.okhttp3:okhttp:jar:4.8.1:compile [INFO] | | +- com.squareup.okio:okio:jar:2.7.0:compile [INFO] | | | - org.jetbrains.kotlin:kotlin-stdlib-common:jar:1.3.70:compile [INFO] | | - org.jetbrains.kotlin:kotlin-stdlib:jar:1.3.72:compile [INFO] | - org.apache.commons:commons-math3:jar:3.6.1:compile [INFO] +- com.google.code.gson:gson:jar:2.8.5:compile [INFO] +- org.json:json:jar:20090211:compile [INFO] +- org.apache.commons:commons-lang3:jar:3.9:compile [INFO] +- com.microsoft.azure:applicationinsights-web-auto:jar:2.5.0:compile [INFO] +- javax.mail:mail:jar:1.5.0-b01:compile [INFO] | - javax.activation:activation:jar:1.1:compile [INFO] +- com.oracle:ojdbc:jar:8:system [INFO] +- org.apache.commons:commons-dbcp2:jar:2.7.0:compile [INFO] | +- org.apache.commons:commons-pool2:jar:2.7.0:compile [INFO] | - commons-logging:commons-logging:jar:1.2:compile [INFO] +- org.junit.jupiter:junit-jupiter:jar:5.4.2:test [INFO] | +- org.junit.jupiter:junit-jupiter-api:jar:5.4.2:test [INFO] | | +- org.apiguardian:apiguardian-api:jar:1.0.0:test [INFO] | | +- org.opentest4j:opentest4j:jar:1.1.1:test [INFO] | | - org.junit.platform:junit-platform-commons:jar:1.4.2:test [INFO] | +- org.junit.jupiter:junit-jupiter-params:jar:5.4.2:test [INFO] | - org.junit.jupiter:junit-jupiter-engine:jar:5.4.2:test [INFO] | - org.junit.platform:junit-platform-engine:jar:1.4.2:test [INFO] +- org.mockito:mockito-core:jar:2.23.4:test [INFO] | +- net.bytebuddy:byte-buddy:jar:1.9.3:test [INFO] | +- net.bytebuddy:byte-buddy-agent:jar:1.9.3:test [INFO] | - org.objenesis:objenesis:jar:2.6:test [INFO] - redis.clients:jedis:jar:3.2.0:compile

I had one other hunch that maybe the applicationinsights library may have been conflicting because it comes from a different non-BOM package

com.microsoft.azure applicationinsights-core 2.6.3

but that turned up nothing. I noticed that had it's own shaded version of "LoggingHandler" which seems to be the problem class right now.

Paizo commented 3 years ago

thank you @alzimmermsft ,

I added the bom and still failing with NoSuchMethodError: io.netty.handler.logging.LoggingHandler.(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;), pom.xml snippet:

    <azure.functions.java.library.version>1.4.2</azure.functions.java.library.version>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.azure</groupId>
                <artifactId>azure-sdk-bom</artifactId>
                <version>1.0.3</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>com.microsoft.azure.functions</groupId>
            <artifactId>azure-functions-java-library</artifactId>
            <version>${azure.functions.java.library.version}</version>
        </dependency>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-messaging-servicebus</artifactId>
        </dependency>
        <!--    http components 5    -->
        <dependency>
            <groupId>org.apache.httpcomponents.client5</groupId>
            <artifactId>httpclient5</artifactId>
            <version>5.0.3</version>
        </dependency>
        <!--   Keyvault secrets     -->
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-security-keyvault-secrets</artifactId>
        </dependency>

        <!--   Keyvault Certificates  -->
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-security-keyvault-certificates</artifactId>
        </dependency>
        <dependency>
            <groupId>com.azure</groupId>
            <artifactId>azure-identity</artifactId>
            <!--   Fix dependency older that 10 years      -->
            <exclusions>
                <exclusion>
                    <groupId>xpp3</groupId>
                    <artifactId>xpp3</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>xpp3</groupId>
            <artifactId>xpp3</artifactId>
            <version>1.1.4c</version>
        </dependency>
        <!--   tentative fix azure libraries NoSuchMethodError/NoClassDef  
        https://github.com/Azure/azure-sdk-for-java/issues/19041   
        https://github.com/Azure/azure-sdk-for-java/issues/22242 -->
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty-http</artifactId>
        </dependency>
        <!--   END - fix azure libraries NoSuchMethodError/NoClassDef  
        https://github.com/Azure/azure-sdk-for-java/issues/19041    
        https://github.com/Azure/azure-sdk-for-java/issues/22242 -->

        <!--    Logging    -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>log4j-over-slf4j</artifactId>
            <version>1.7.30</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.3</version>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.20</version>
            <scope>provided</scope>
        </dependency>

        <!-- Test -->
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>5.7.1</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>3.9.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-junit-jupiter</artifactId>
            <version>3.9.0</version>
            <scope>test</scope>
        </dependency>
        <!--  Integration test -->
        <dependency>
            <groupId>org.mock-server</groupId>
            <artifactId>mockserver-netty</artifactId>
            <version>5.11.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mock-server</groupId>
            <artifactId>mockserver-client-java</artifactId>
            <version>5.11.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mock-server</groupId>
            <artifactId>mockserver-junit-jupiter</artifactId>
            <version>5.11.2</version>
            <scope>test</scope>
        </dependency>
        <!--  testing code which uses java.lang.System (ie: System.getenv()) -->
        <dependency>
            <groupId>com.github.stefanbirkner</groupId>
            <artifactId>system-lambda</artifactId>
            <version>1.2.0</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>uk.org.webcompere</groupId>
            <artifactId>system-stubs-jupiter</artifactId>
            <version>1.2.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

logs:

6f0f4a663f4f','level':'Error','levelId':4,'processId':18,'exceptionDetails':'Microsoft.Azure.WebJobs.Host.FunctionInvocationException : Exception while executing function: Functions.ArchiveMessageFunction ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: Failure\nException: NoSuchMethodError: io.netty.handler.logging.LoggingHandler.<init>(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V\nStack: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodExecutorImpl.execute(JavaMethodExecutorImpl.java:54)\n\tat com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:57)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)\n\tat com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)\n\tat com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.NoSuchMethodError: io.netty.handler.logging.LoggingHandler.<init>(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V\n\tat reactor.netty.transport.logging.ReactorNettyLoggingHandler.<init>(ReactorNettyLoggingHandler.java:63)\n\tat reactor.netty.transport.logging.AdvancedByteBufFormat.toLoggingHandler(AdvancedByteBufFormat.java:109)\n\tat reactor.netty.http.client.HttpClientConfig.<clinit>(HttpClientConfig.java:635)\n\tat reactor.netty.http.client.HttpClientConnect.<init>(HttpClientConnect.java:84)\n\tat reactor.netty.http.client.HttpClient.create(HttpClient.java:391)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17)\n\tat com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:50)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:40)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildAsyncClient(CertificateClientBuilder.java:167)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildClient(CertificateClientBuilder.java:109)\n\tat com.abnamro.ibii.repository.AzureCertificateRepository.<init>

if I add the netty-all dependency then the exception is different, NoClassDefFoundError reactor.netty.http.client.HttpClientConfig:

        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-all</artifactId>
        </dependency>

Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException : Result: Failure\nException: NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig\nStack: java.lang.reflect.InvocationTargetException\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)\n\tat com.microsoft.azure.functions.worker.broker.JavaMethodExecutorImpl.execute(JavaMethodExecutorImpl.java:54)\n\tat com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:57)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)\n\tat com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)\n\tat com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)\n\tat com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)\n\tat java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\nCaused by: java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig\n\tat reactor.netty.http.client.HttpClientConnect.<init>(HttpClientConnect.java:84)\n\tat reactor.netty.http.client.HttpClient.create(HttpClient.java:391)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97)\n\tat com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17)\n\tat com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:50)\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:40)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildAsyncClient(CertificateClientBuilder.java:167)\n\tat com.azure.security.keyvault.certificates.CertificateClientBuilder.buildClient(CertificateClientBuilder.java:109)\n\tat com.abnamro.ibii.repository.AzureCertificateRepository.<init>

Paizo commented 3 years ago

gave it a try without the netty-all and added all the netty/reactor dependencies I can see in the dependency:tree like so:

        <dependency>
            <groupId>io.projectreactor</groupId>
            <artifactId>reactor-core</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty-core</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty-http</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.netty</groupId>
            <artifactId>reactor-netty-http-brave</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec-socks</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-handler-proxy</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-transport-native-epoll</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-transport-native-kqueue</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-tcnative-boringssl-static</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec-http2</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-transport-native-unix-common</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-resolver-dns</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec-dns</artifactId>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-codec-http2</artifactId>
        </dependency>

the error is still the same: java.lang.NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig

derekg52 commented 3 years ago

For my scenario, I reverted back to a pom from a few months ago to take a fresh look. What I noticed is when I first added the Cosmos artifact (with a defined version like 4.15), that's when I saw these errors again. After deleting my local m2 and starting with an old version (4.1) in this case, it worked OK.

What I noticed was Cosmos required a newer netty than my other azure dependencies and from viewing the dependency tree, bumped up netty from 4.1.42 to 4.1.49.

For the moment, I think I can work with this but I have some more analysis to do to see where it finally breaks, and what version of netty that corresponds to.

derekg52 commented 3 years ago

I spent a few more hours wrestling with various dependency inclusions of parts of netty and reactor netty with no luck.

What worked for me was following the guide of switching from using Netty to OkHTTP per (https://docs.microsoft.com/en-us/azure/developer/java/sdk/http-client-pipeline)

It immediately worked with no fuss, I'm going to continue exploring this route. Hopefully this information helps someone else out.

alzimmermsft commented 3 years ago

Thank you for the additional information @Paizo and @derekg52. For these applications, are they running locally or in Azure Functions? Would it be possible to do a quick run locally to determine if the environment has any impact on dependency resolution. For example, is a custom class loader being used which resolves the Netty dependencies to another version compared to what Maven dependency:tree is reporting as the resolution version.

derekg52 commented 3 years ago

Hey @alzimmermsft - I'm attempting to get my function running again locally though in my Azure dev environment it runs but failed when I moved the artifact over to test. I noticed that in my case, I think Cosmos requires Netty and using OkHTTP doesn't seem to work when it's directed to. Interestingly enough, I was able to run some of the Java mains through eclipse OK though it fails when invoking the API endpoint through the function.

What I have to have printed out at this point it looks like is comparing what the classpath eclipse is running with versus the tree from maven when running the function. I'll get back to you.

Paizo commented 3 years ago

thanks @alzimmermsft and @derekg52 ,

tested both locally and on azure function, still have to give a try to okhttp. This function has an infinite retry and locally I can clearly see it failing for NoSuchMethodError: io.netty.handler.logging.LoggingHandler at the first invocation and the subsequent retries with NoClassDefFoundError: Could not initialize class reactor.netty.http.client.HttpClientConfig

derekg52 commented 3 years ago

@Paizo yep that's where I am, endless loops chasing netty messages even when I try to exclude them to use the okhttp implementation:

[2021-06-14T23:47:06.817Z] Caused by: java.lang.NoSuchMethodError: io.netty.handler.logging.LoggingHandler.(Ljava/lang/String;Lio/netty/handler/logging/LogLevel;Lio/netty/handler/logging/ByteBufFormat;)V [2021-06-14T23:47:06.818Z] at reactor.netty.transport.logging.ReactorNettyLoggingHandler.(ReactorNettyLoggingHandler.java:63) [2021-06-14T23:47:06.820Z] at reactor.netty.transport.logging.AdvancedByteBufFormat.toLoggingHandler(AdvancedByteBufFormat.java:109) [2021-06-14T23:47:06.823Z] at reactor.netty.http.client.HttpClientConfig.(HttpClientConfig.java:635) [2021-06-14T23:47:06.824Z] at reactor.netty.http.client.HttpClientConnect.(HttpClientConnect.java:84) [2021-06-14T23:47:06.826Z] at reactor.netty.http.client.HttpClient.create(HttpClient.java:391) [2021-06-14T23:47:06.827Z] at com.azure.core.http.netty.NettyAsyncHttpClientBuilder.build(NettyAsyncHttpClientBuilder.java:97) [2021-06-14T23:47:06.828Z] at com.azure.core.http.netty.NettyAsyncHttpClientProvider.createInstance(NettyAsyncHttpClientProvider.java:17) [2021-06-14T23:47:06.830Z] at com.azure.core.implementation.http.HttpClientProviders.createInstance(HttpClientProviders.java:58) [2021-06-14T23:47:06.831Z] at com.azure.core.http.HttpClient.createDefault(HttpClient.java:50) [2021-06-14T23:47:06.835Z] at com.azure.core.http.HttpClient.createDefault(HttpClient.java:40) [2021-06-14T23:47:06.836Z] at com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:62) [2021-06-14T23:47:06.837Z] at com.azure.data.appconfiguration.ConfigurationClientBuilder.buildAsyncClient(ConfigurationClientBuilder.java:203)

I even stood up a new function in VS code and slowly dropped my pieces into it with the same results. I was thinking maybe something from my original pom was setup wrong or something along those lines but it's been consistent whether it was VS code or my Eclipse one.

alzimmermsft commented 3 years ago

Hi @Paizo and @derekg52, for using OkHttp as the HTTP networking layer instead of Netty, do all of the included Azure SDKs include the azure-core-http-netty exclusion? Based on your dependency trees:

@Paizo 

<dependencies>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-secrets</artifactId>
        <version>4.2.8</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-certificates</artifactId>
        <version>4.1.8</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.3.1</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
</dependencies>

@derekg52 

<dependencies>
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-storage-blob</artifactId>
        <version>12.11.1</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-cosmos</artifactId>
        <version>4.15.0</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.3.0</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-security-keyvault-secrets</artifactId>
        <version>4.2.8</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-data-appconfiguration</artifactId>
        <version>1.1.12</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core-http-netty</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
</dependencies>
alzimmermsft commented 3 years ago

Hi @Paizo and @derekg52, which version of Java is being used by Azure Functions? If you are using Java 8 could you run your application with the FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS environment variable set to true.

derekg52 commented 3 years ago

@alzimmermsft DING DING DING! That was it, thank you a million times, who knew one little setting like that would do it? I'm going to do some more unit testing on my end but I was able to use the BOM definition and also it uses netty still (I removed all the exclusions and the okhttp implementation).

derekg52 commented 3 years ago

Just to put one more stake in this, I checked my configuration and my dev environment function in Azure is Java 11 and the test one (the one that didn't work), was set to Java 8.

Paizo commented 3 years ago

yep, that was it. Thank you

Currently using java 1.8 both locally and on azure; now with the bom and no extra libraries such as netty the flag solve the issue. Is this documented somewhere? Are there other similar flags we need to pay attention?

alzimmermsft commented 3 years ago

At this time it isn't documented anywhere aside from source code in the azure-functions-java-library library. I'll follow-up to getting this documented in a better location.

ghost commented 3 years ago

Hi @Paizo. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

ghost commented 3 years ago

Hi @Paizo, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.

AlejandroBlanco commented 3 years ago

@alzimmermsft DING DING DING! That was it, thank you a million times, who knew one little setting like that would do it? I'm going to do some more unit testing on my end but I was able to use the BOM definition and also it uses netty still (I removed all the exclusions and the okhttp implementation).

Can you elaborate how you are using that app setting. I have added it in my local.settings.json like: "FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS": true and I have added it too in my pom like:

            <appSettings>
                <property>
                    <name>FUNCTIONS_EXTENSION_VERSION</name>
                    <value>~3</value>
                </property>
                <property>
                    <name>FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS</name>
                    <value>1</value>
                </property>
            </appSettings>

under azure functions maven plugin.

Still, i have no results...

Paizo commented 3 years ago

@AlejandroBlanco I'm using this in local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    ...
    "FUNCTIONS_WORKER_RUNTIME": "java",
    "JAVA_HOME": "/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home",
    "FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS": "true",
    ... 
  }
}

no changes in the pom.xml but I pass them as app settings during deployment: image

derekg52 commented 3 years ago

@AlejandroBlanco I'm using this in local.settings.json

{
  "IsEncrypted": false,
  "Values": {
    ...
    "FUNCTIONS_WORKER_RUNTIME": "java",
    "JAVA_HOME": "/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home",
    "FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS": "true",
    ... 
  }
}

no changes in the pom.xml but I pass them as app settings during deployment: image

@AlejandroBlanco - the change I made was from going from Java 8 to 11. That was what made it magically work for me. I didn't try the FUNCTIONS_WORKER_JAVA_LOAD_APP_LIBS setting, although from the comments from @alzimmermsft that should work as well if you keep your stack on Java 8.