search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.25k stars 1.93k forks source link

[BUG] Azure SQL ActiveDirectoryManagedIdentity authentication with sleuth #40273

Open alekseistepanovvl opened 1 month ago

alekseistepanovvl commented 1 month ago

Describe the bug Our Spring Boot 2.18 application, utilizing the spring-boot-starter-data-jpa, is configured with the mssql-jdbc version 12.6.1.jre11, azure-identity version 1.12.0, msal4j version 1.15.0, spring-cloud-starter-sleuth version 3.1.11 dependencies. Java version 11

The application, dockerized with the base image openjdk:11, is earmarked for deployment on Azure Kubernetes Service (AKS). A crucial requirement mandates that the application establish a connection to Azure SQL utilizing ActiveDirectoryManagedIdentity authentication.

Interestingly, the application connects to the database successfully in the absence of the spring-cloud-starter-sleuth dependency. However, upon its inclusion, the application encounters a freezing behavior(Doesn't connect to db and no exceptions in log).

Exception or Stack Trace I've reproduced Managed Identity behavior on my environment, by putting environment variables. I noticed several blocked threads, that I think responsible for bug. Print here thread trace: `"main" #1 prio=5 os_prio=31 cpu=1854.71ms elapsed=1248.18s tid=0x000000014300cc00 nid=0x1803 waiting on condition [0x000000016f307000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@17.0.9/Native Method)

` "ForkJoinPool.commonPool-worker-1" #25 daemon prio=5 os_prio=31 cpu=25.30ms elapsed=1239.41s tid=0x0000000141a3e800 nid=0x9e03 waiting for monitor entry [0x0000000173fdc000] java.lang.Thread.State: BLOCKED (on object monitor) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getSingletonFactoryBeanForTypeCheck(AbstractAutowireCapableBeanFactory.java:1006)

To Reproduce You can use this repository to reproduce the issue. To imitate Managed Identity behavior add properties from readme to env variables. https://github.com/alekseistepanovvl/demo-service main branch To reproduce try run application from repo with env variables.

Code Snippet I noticed that Sleuth removal resolves fixes and issues. Therefore, I believe there may be an issue with the interaction between Azure Identity and Sleuth. 

at org.springframework.cloud.sleuth.instrument.reactor.ReactorSleuth.lambda$scopePassingOnScheduleHook$7(ReactorSleuth.java:323)
at org.springframework.cloud.sleuth.instrument.reactor.ReactorSleuth$$Lambda$556/0x00000070004deb10.apply(Unknown Source)
at reactor.core.scheduler.Schedulers.onSchedule(Schedulers.java:1017)
at reactor.core.scheduler.Schedulers.directSchedule(Schedulers.java:1380)
at reactor.core.scheduler.ParallelScheduler.schedule(ParallelScheduler.java:243)
at reactor.core.scheduler.Schedulers$CachedScheduler.schedule(Schedulers.java:1308)
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:320)
at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onNext(MonoCacheTime.java:354)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
at reactor.core.publisher.MonoCallable.subscribe(MonoCallable.java:62)
at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
at reactor.core.publisher.Mono.subscribe(Mono.java:4475)
at reactor.core.publisher.Mono.subscribeWith(Mono.java:4605)
at reactor.core.publisher.Mono.toFuture(Mono.java:5010)
at com.azure.identity.implementation.IdentityClientBase.lambda$getManagedIdentityConfidentialClient$3(IdentityClientBase.java:424)
at com.azure.identity.implementation.IdentityClientBase$$Lambda$1047/0x000000700082fcb8.apply(Unknown Source)

Expected behavior Connection is getting set using authentication=ActiveDirectoryManagedIdentity with and without sleuth dependency.

Screenshots If applicable, add screenshots to help explain your problem.

Setup (please complete the following information):

Additional context I guess issue in org.springframework.cloud.sleuth.instrument.reactor.ReactorSleuth.lambda$scopePassingOnScheduleHook. Called it in debug and it's calculation seems blocked the thread.

Screenshot 2024-05-20 at 22 52 22

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

github-actions[bot] commented 1 month ago

@billwert @g2vinay

github-actions[bot] commented 1 month ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

alekseistepanovvl commented 1 month ago

Looks like deadlock:

"main@1" prio=5 tid=0x1 nid=NA waiting
  java.lang.Thread.State: WAITING
     blocks ForkJoinPool.commonPool-worker-1@9633
      at jdk.internal.misc.Unsafe.park(Unsafe.java:-1)
"ForkJoinPool.commonPool-worker-1@9633" daemon prio=5 tid=0x1b nid=NA waiting for monitor entry
  java.lang.Thread.State: BLOCKED
     waiting for main@1 to release lock on <0x312b> (a java.util.concurrent.ConcurrentHashMap)
      at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.getSingletonFactoryBeanForTypeCheck(AbstractAutowireCapableBeanFactory.java:1006)