search

AzureAD / microsoft-authentication-library-for-java

Microsoft Authentication Library (MSAL) for Java http://aka.ms/aadv2
MIT License
285 stars 142 forks source link

[Bug] NoSuchMethodError (ConfidentialClientApplication$Builder.executorService): mssql-jdbc + msal4j (since v1.15.0) #806

Closed gavvvr closed 3 months ago

gavvvr commented 6 months ago

Library version used

1.15.0

Java version

17.0.6

Scenario

Other - please specify

Is this a new or an existing app?

None

Issue description and reproduction steps

There is Java app connecting to MSSQL db over something called "Microsoft Entra". After an update of msal4j from v1.14.x to v1.15.0, there the jdbc driver throws NoSuchMethodError while authenticating.

I reproduced the problem with a the below minimal code snippet

Relevant code snippets

dependencies {
    implementation("com.microsoft.sqlserver:mssql-jdbc:12.6.1.jre11")
    implementation("com.microsoft.azure:msal4j:1.15.0") // <-- the previous 1.14.3 worked well
    implementation("org.slf4j:slf4j-simple:2.0.12")
}
import com.microsoft.sqlserver.jdbc.SQLServerDataSource

fun main() {
    Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
    val ds = SQLServerDataSource()
    ds.url = "jdbc:sqlserver://ms-sql-server-host:1433;database=some-db;Authentication=ActiveDirectoryServicePrincipal"
    ds.user = "?"
    ds.setPassword("?")
    val con = ds.getConnection()
    con.close()
}

Expected behavior

No breaking change expected on minor version upgrade.

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

1.14.3

Solution and workarounds

Not to upgrade to msal4j v1.15.0 and ignore it by Renovate bot

gavvvr commented 6 months ago

Most probably it's related to 450757afc5dad refactoring.

Avery-Dunn commented 6 months ago

Hello @gavvvr : Could you post the exact error/stack trace you're getting? Is it the 'executorService' referenced in your title that's causing the NoSuchMethodError?

Also, it looks like you're using the latest mssql-jdbc:12.6.1.jre11, but that uses msa4lj 1.14.1 . Based on that code snippet I assume you're using Scala, but I'm not too familiar with it or that build system. I assume during runtime it's using version 1.15.0 because of the explicit msal4j dependency rather than the transient 1.14.1?

gavvvr commented 6 months ago

Hi @Avery-Dunn

It's the simplest Kotlin project built by Gradle. The combination of mssql-jdbc:12.6.1.jre11 + msal4j:1.15.0. Upgrading msal4j from v1.14.3 to v1.15.0 I don't expect backward compatibility issues.

The complete stacktace is below:

Click me ```text Exception in thread "main" java.lang.NoSuchMethodError: 'com.microsoft.aad.msal4j.AbstractClientApplicationBase$Builder com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.executorService(java.util.concurrent.ExecutorService)'   at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getSqlFedAuthTokenPrincipal(SQLServerMSAL4JUtils.java:137)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.getFedAuthToken(SQLServerConnection.java:6028)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.onFedAuthInfo(SQLServerConnection.java:5963)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.processFedAuthInfo(SQLServerConnection.java:5797)   at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onFedAuthInfo(tdsparser.java:322)   at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:130)   at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:42)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:6855)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:5402)   at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:5334)   at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7739)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:4384)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3823)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3348)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3179)   at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1953)   at com.microsoft.sqlserver.jdbc.SQLServerDataSource.getConnectionInternal(SQLServerDataSource.java:1550)   at com.microsoft.sqlserver.jdbc.SQLServerDataSource.getConnection(SQLServerDataSource.java:99)   at org.example.AppKt.main(App.kt:11)   at org.example.AppKt.main(App.kt) ```
Avery-Dunn commented 5 months ago

Hello @gavvvr : We're having trouble reproducing this error, however other customers have reported something similar: one after upgrading to 1.15.0 like yourself, and one a few weeks ago after upgrading to 1.14.0

The person who upgraded to 1.15.0 was able to get rid of the issue by purging their local Maven repo, ensuring that fresh versions of every dependency was downloaded the next time they built their package. Could you try deleting any local versions of msla4j (in Maven it's the '.m2' folder, not sure for Gradle), and letting us know if you still have this 'NoSuchMethod' issue?

gavvvr commented 5 months ago

Hi @Avery-Dunn

I use Gradle instead of Maven. The build is as simple as the following build.gradle.kts:

plugins {
    application
    kotlin("jvm") version "1.9.23"
}

repositories {
    mavenCentral()
}

application {
    mainClass.set("com.example.AppKt")
}

dependencies {
    implementation("com.microsoft.sqlserver:mssql-jdbc:12.6.1.jre11")
    implementation("com.microsoft.azure:msal4j:1.15.0")
    implementation("org.slf4j:slf4j-simple:2.0.12")
}

and the app gets built on CI on a fresh ephemeral build agents. The main method for obtaining jdbc connection was listed above. It's a hello-world setup.

To reproduce an issue, you need a db using MS Entra authentication method.

blutorange commented 5 months ago

Well, we did have the the same issue. We pin the exact versions of many dependencies (due to the dependencyConvergence rule), and we pinned msal4j to 1.15.0 (to stay up-to-date, and higher minor versions should be compatible).

Due this issue, we changed msal4j to 1.14.3, which gets rid of the error.

The cause of the error seems quite straight-forward, not requiring reproduction. As already mentioned by @gavvvr, version 1.15.0 of msal4j removes the executorService method from AbstractClientApplicationBase#Builder and moves it to AbstractApplicationBase$Builder. If you take a look at that stack trace also posted by gavvvr, that method is called by com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getSqlFedAuthTokenPrincipal(SQLServerMSAL4JUtils.java:137), which will fail since that method was removed.

gavvvr commented 5 months ago

Hi @Avery-Dunn

The problem is obvious to me now. After moving of public T executorService(ExecutorService val) method from AbstractClientApplicationBase.Builder<T extends AbstractClientApplicationBase.Builder<T>> into the AbstractApplicationBase.Builder<T extends AbstractApplicationBase.Builder<T>>, the executorService-method signature has changed.

Previously it returned AbstractClientApplicationBase$Builder, but since v1.15.0 it returns AbstractApplicationBase$Builder.

It's a backward-incompatible change in runtime for mssql-jdbc:12.6.1 compiled against of msal4j:1.14.1 and expecting old method signature. For compile-time there is no issue though...

Here is the code for reproducing the problem. Supposed you have 2 Maven modules: app and lib (Gradle's subprojects in my case).

Here is the Builder.java from lib:

package org.example.lib;

public class Builder extends AbstractBuilder<Builder> {
    @Override
    Builder self() {
        return this;
    }
}

abstract class AbstractBuilder<T extends AbstractBuilder<T>> /*extends BuilderBase<T>*/ {
    public T executorService() {
        return self();
    }

    abstract T self();
}

abstract class BuilderBase<T extends BuilderBase<T>> {
    public T executorService() {
        return self();
    }

    abstract T self();
}

And the App.java in app is:

package org.example.app;

import org.example.lib.Builder;

public class App {
    public static void main(String[] args) {
        var builder = new Builder();
        System.out.println(builder.executorService());
    }
}

If you build the app without extending BuilderBase, and then build an updated lib.jar with with uncommented extends BuilderBase<T> and commented out AbstractBuilder#executorService/AbstractBuilder#self methods, then you will get a:

Exception in thread "main" java.lang.NoSuchMethodError: 'org.example.lib.AbstractBuilder org.example.lib.Builder.executorService()'
        at org.example.app.App.main(App.java:10)

I've created a repository with README.md which reproduces the problem: https://github.com/gavvvr/msal4j-806

jdafont commented 4 months ago

I have encountered what sounds like the same issue as @gavvvr where I'm using the latest com.microsoft.azure.kusto.kusto-data:5.0.5, which is compiled against com.microsoft.azure.msal4j:1.13.10. After updating the azure-sdk-bom to the latest (which uses 1.15.0), I am getting this error:

java.lang.NoSuchMethodError: 'com.microsoft.aad.msal4j.AbstractClientApplicationBase$Builder com.microsoft.aad.msal4j.ConfidentialClientApplication$Builder.httpClient(com.microsoft.aad.msal4j.IHttpClient)'
    at com.microsoft.azure.kusto.data.auth.ApplicationKeyTokenProvider.getClientApplication(ApplicationKeyTokenProvider.java:31)
    at com.microsoft.azure.kusto.data.auth.ConfidentialAppTokenProviderBase.initializeWithCloudInfo(ConfidentialAppTokenProviderBase.java:38)
    at com.microsoft.azure.kusto.data.auth.CloudDependentTokenProviderBase.initialize(CloudDependentTokenProviderBase.java:41)
    at com.microsoft.azure.kusto.data.auth.TokenProviderBase.acquireAccessToken(TokenProviderBase.java:30)
    at com.microsoft.azure.kusto.data.ClientImpl.generateIngestAndCommandHeaders(ClientImpl.java:405)
    at com.microsoft.azure.kusto.data.ClientImpl.executeToJsonResult(ClientImpl.java:213)
    at com.microsoft.azure.kusto.data.ClientImpl.executeImpl(ClientImpl.java:173)
    at com.microsoft.azure.kusto.data.ClientImpl.lambda$execute$0(ClientImpl.java:122)
    at com.microsoft.azure.kusto.data.instrumentation.MonitoredActivity.invoke(MonitoredActivity.java:33)
    at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:121)
    at com.microsoft.azure.kusto.data.ClientImpl.execute(ClientImpl.java:116)

Edit: Just confirmed that downgrading to the expected version works fine, and that the httpClient method (property on base class with annotation) was removed in 1.15.0, breaking the Kusto data library. So, yeah the same breaking of backwards compatability in minor version release.

CodeHipster commented 4 months ago

Running into the same issue when upgrading our dependencies.

mssql-jdbc(12.4.2) comes with spring-boot(3.2.6) msal4j(1.15.0) comes from azure-identity(1.12.0) which comes from spring-cloud-azure-dependencies(5.12.0)

So this bug actually makes the version mapping incorrect at https://github.com/Azure/azure-sdk-for-java/wiki/Spring-Versions-Mapping

spring-boot 3.2.x is not compatible with 5.x.x when using mssql+entra id auth.

Staying with spring-cloud-azure-dependencies:5.11.0 does not have this problem

straakaa commented 3 months ago

Hi, I have the same issue: Using azure-identity with mssql-jdbc. We need update azure-identity to version 1.12.2 due to a vulnerability CVE-2024-35255 and currently there is no compatible version of msal4j with with any mssql-jdbc library.

gonzdae commented 3 months ago

Hi, I have a different issue but it's related to upgrading to v1.15.0 and 1.15.1. The error I'm getting is

Correct the classpath of your application so that it contains compatible versions of the classes com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils and com.microsoft.aad.msal4j.PublicClientApplication$Builder.

Purged my local directory and all, nothing works.

i1888 commented 3 months ago

Reproduce on com.microsoft.azure:msal4j:1.15.1

image

"C:\Program Files\Microsoft\jdk-21.0.3.9-hotspot\bin\java.exe" "-javaagent:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2024.1.3\lib\idea_rt.jar=54324:C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2024.1.3\bin" -Dfile.encoding=UTF-8 -Dsun.stdout.encoding=UTF-8 -Dsun.stderr.encoding=UTF-8 -classpath C:\Users\yuanzhao\IdeaProjects\quick01\target\classes;C:\Users\yuanzhao.m2\repository\com\azure\azure-identity\1.12.2\azure-identity-1.12.2.jar;C:\Users\yuanzhao.m2\repository\com\azure\azure-core\1.49.1\azure-core-1.49.1.jar;C:\Users\yuanzhao.m2\repository\com\azure\azure-xml\1.0.0\azure-xml-1.0.0.jar;C:\Users\yuanzhao.m2\repository\com\fasterxml\jackson\core\jackson-annotations\2.13.5\jackson-annotations-2.13.5.jar;C:\Users\yuanzhao.m2\repository\com\fasterxml\jackson\core\jackson-core\2.13.5\jackson-core-2.13.5.jar;C:\Users\yuanzhao.m2\repository\com\fasterxml\jackson\core\jackson-databind\2.13.5\jackson-databind-2.13.5.jar;C:\Users\yuanzhao.m2\repository\com\fasterxml\jackson\datatype\jackson-datatype-jsr310\2.13.5\jackson-datatype-jsr310-2.13.5.jar;C:\Users\yuanzhao.m2\repository\org\slf4j\slf4j-api\1.7.36\slf4j-api-1.7.36.jar;C:\Users\yuanzhao.m2\repository\io\projectreactor\reactor-core\3.4.38\reactor-core-3.4.38.jar;C:\Users\yuanzhao.m2\repository\org\reactivestreams\reactive-streams\1.0.4\reactive-streams-1.0.4.jar;C:\Users\yuanzhao.m2\repository\com\azure\azure-core-http-netty\1.15.1\azure-core-http-netty-1.15.1.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-handler\4.1.110.Final\netty-handler-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-resolver\4.1.110.Final\netty-resolver-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport\4.1.110.Final\netty-transport-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-handler-proxy\4.1.110.Final\netty-handler-proxy-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-codec-socks\4.1.110.Final\netty-codec-socks-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-buffer\4.1.110.Final\netty-buffer-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-codec\4.1.110.Final\netty-codec-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-codec-http\4.1.110.Final\netty-codec-http-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-codec-http2\4.1.110.Final\netty-codec-http2-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport-native-unix-common\4.1.110.Final\netty-transport-native-unix-common-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport-native-epoll\4.1.110.Final\netty-transport-native-epoll-4.1.110.Final-linux-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport-classes-epoll\4.1.110.Final\netty-transport-classes-epoll-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport-native-kqueue\4.1.110.Final\netty-transport-native-kqueue-4.1.110.Final-osx-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-transport-classes-kqueue\4.1.110.Final\netty-transport-classes-kqueue-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-classes\2.0.65.Final\netty-tcnative-classes-2.0.65.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final-linux-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final-linux-aarch_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final-osx-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final-osx-aarch_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-tcnative-boringssl-static\2.0.65.Final\netty-tcnative-boringssl-static-2.0.65.Final-windows-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\projectreactor\netty\reactor-netty-http\1.0.45\reactor-netty-http-1.0.45.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-resolver-dns\4.1.109.Final\netty-resolver-dns-4.1.109.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-codec-dns\4.1.109.Final\netty-codec-dns-4.1.109.Final.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-resolver-dns-native-macos\4.1.109.Final\netty-resolver-dns-native-macos-4.1.109.Final-osx-x86_64.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-resolver-dns-classes-macos\4.1.109.Final\netty-resolver-dns-classes-macos-4.1.109.Final.jar;C:\Users\yuanzhao.m2\repository\io\projectreactor\netty\reactor-netty-core\1.0.45\reactor-netty-core-1.0.45.jar;C:\Users\yuanzhao.m2\repository\io\netty\netty-common\4.1.110.Final\netty-common-4.1.110.Final.jar;C:\Users\yuanzhao.m2\repository\com\azure\azure-json\1.1.0\azure-json-1.1.0.jar;C:\Users\yuanzhao.m2\repository\com\microsoft\azure\msal4j\1.15.1\msal4j-1.15.1.jar;C:\Users\yuanzhao.m2\repository\com\nimbusds\oauth2-oidc-sdk\11.9.1\oauth2-oidc-sdk-11.9.1.jar;C:\Users\yuanzhao.m2\repository\com\github\stephenc\jcip\jcip-annotations\1.0-1\jcip-annotations-1.0-1.jar;C:\Users\yuanzhao.m2\repository\com\nimbusds\content-type\2.3\content-type-2.3.jar;C:\Users\yuanzhao.m2\repository\com\nimbusds\lang-tag\1.7\lang-tag-1.7.jar;C:\Users\yuanzhao.m2\repository\com\nimbusds\nimbus-jose-jwt\9.37.3\nimbus-jose-jwt-9.37.3.jar;C:\Users\yuanzhao.m2\repository\net\minidev\json-smart\2.5.0\json-smart-2.5.0.jar;C:\Users\yuanzhao.m2\repository\net\minidev\accessors-smart\2.5.0\accessors-smart-2.5.0.jar;C:\Users\yuanzhao.m2\repository\org\ow2\asm\asm\9.3\asm-9.3.jar;C:\Users\yuanzhao.m2\repository\com\microsoft\azure\msal4j-persistence-extension\1.3.0\msal4j-persistence-extension-1.3.0.jar;C:\Users\yuanzhao.m2\repository\net\java\dev\jna\jna\5.13.0\jna-5.13.0.jar;C:\Users\yuanzhao.m2\repository\net\java\dev\jna\jna-platform\5.6.0\jna-platform-5.6.0.jar;C:\Users\yuanzhao.m2\repository\com\microsoft\sqlserver\mssql-jdbc\12.7.0.jre11-preview\mssql-jdbc-12.7.0.jre11-preview.jar org.example.App Hello World! Exception in thread "main" java.lang.NoSuchMethodError: 'com.microsoft.aad.msal4j.AbstractClientApplicationBase$Builder com.microsoft.aad.msal4j.PublicClientApplication$Builder.executorService(java.util.concurrent.ExecutorService)' at com.microsoft.sqlserver.jdbc.SQLServerMSAL4JUtils.getSqlFedAuthToken(SQLServerMSAL4JUtils.java:100) at com.microsoft.sqlserver.jdbc.SQLServerConnection.getFedAuthToken(SQLServerConnection.java:6034) at com.microsoft.sqlserver.jdbc.SQLServerConnection.onFedAuthInfo(SQLServerConnection.java:6001) at com.microsoft.sqlserver.jdbc.SQLServerConnection.processFedAuthInfo(SQLServerConnection.java:5835) at com.microsoft.sqlserver.jdbc.TDSTokenHandler.onFedAuthInfo(tdsparser.java:346) at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:130) at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:42) at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:6894) at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:5440) at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:5372) at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7775) at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:4397) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3834) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:3391) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:3200) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1974) at com.microsoft.sqlserver.jdbc.SQLServerDataSource.getConnectionInternal(SQLServerDataSource.java:1550) at com.microsoft.sqlserver.jdbc.SQLServerDataSource.getConnection(SQLServerDataSource.java:99) at org.example.App.main(App.java:24)

Process finished with exit code 1

Mr-Vinti commented 3 months ago

Hi guys,

The changes on PR #828 should hopefully fix all of these issues. Let's wait till the fix is released and then see if the issue still reproduces.

i1888 commented 3 months ago

Hi guys,

The changes on PR #828 should hopefully fix all of these issues. Let's wait till the fix is released and then see if the issue still reproduces.

Hello @Mr-Vinti do you know how long it usually takes to be released?

Avery-Dunn commented 3 months ago

Hello all, sorry for the long delay on getting this fixed but version 1.16.0 has now been released to fix this issue. The methods that were moved have been restored to their original class while keeping the new ones in place, so it should be compatible with both pre- and post-1.15.0 versions.

If anyone is still getting this error or having similar issues feel free to re-open this thread or start a new one.