Deadlock while starting gateway application on Kubernetes

[HJK181]

Describe the bug The gateway runs into a deadlock when started inside kubernetes. Thread dump:

"main" prio=0 tid=0x0 nid=0x0 waiting on condition
     java.lang.Thread.State: WAITING
 on java.util.concurrent.CountDownLatch$Sync@2e4f8c0a
    at java.base@17.0.5/jdk.internal.misc.Unsafe.park(Native Method)
    at java.base@17.0.5/java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
    at java.base@17.0.5/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:715)
    at java.base@17.0.5/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1047)
    at java.base@17.0.5/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:230)
    at app//reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:87)
    at app//reactor.core.publisher.Flux.blockLast(Flux.java:2747)
    at app//org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.lambda$onApplicationEvent$0(WeightCalculatorWebFilter.java:134)
    at app//org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter$$Lambda$1703/0x0000000801692e90.accept(Unknown Source)
    at app//org.springframework.beans.factory.support.DefaultListableBeanFactory$DependencyObjectProvider.ifAvailable(DefaultListableBeanFactory.java:2070)
    at app//org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.onApplicationEvent(WeightCalculatorWebFilter.java:134)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
    at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:413)
    at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:370)
    at app//org.springframework.cloud.gateway.route.RouteRefreshListener.reset(RouteRefreshListener.java:73)
    at app//org.springframework.cloud.gateway.route.RouteRefreshListener.onApplicationEvent(RouteRefreshListener.java:54)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
    at app//org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
    at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:413)
    at app//org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:370)
    at app//org.springframework.cloud.kubernetes.commons.discovery.KubernetesDiscoveryClientHealthIndicatorInitializer.postConstruct(KubernetesDiscoveryClientHealthIndicatorInitializer.java:48)
    at java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base@17.0.5/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base@17.0.5/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base@17.0.5/java.lang.reflect.Method.invoke(Method.java:568)
    at app//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:425)
    at app//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:369)
    at app//org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:193)
    at app//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:419)
    at app//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1762)
    at app//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
    at app//org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
    at app//org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326)
    at app//org.springframework.beans.factory.support.AbstractBeanFactory$$Lambda$455/0x0000000800ee0468.getObject(Unknown Source)
    at app//org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
    at app//org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
    at app//org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
    at app//org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
    at app//org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:917)
    at app//org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:584)
    at app//org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66)
    at app//org.springframework.boot.SpringApplication.refresh(SpringApplication.java:732)
    at app//org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434)
    at app//org.springframework.boot.SpringApplication.run(SpringApplication.java:310)
    at app//org.springframework.boot.SpringApplication.run(SpringApplication.java:1304)
    at app//org.springframework.boot.SpringApplication.run(SpringApplication.java:1293)
    at app//io.demo.GatewayApplication.main(GatewayApplication.java:10)

Setting spring.cloud.gateway.predicate.weight.enabled=false works as a workaround. However, it should not be the default behavior for the gateway to fail with the default settings.

Versions used:

• spring-cloud-gateway:4.0.4
• spring-cloud-kubernetes.commons:3.0.2

[liangyuanpeng]

Have some code to reproduce it ?

[HJK181]

Hm, I have a pretty simple configuration file

spring:
      cloud:
        kubernetes:
          client:
            namespace: ui
          discovery:
            enabled: true
          leader:
            config-map-name: gateway-leader
            role: active_user_gauge
        gateway:
          predicate:
            weight:
              enabled: false
          httpclient:
            connect-timeout: 2000
          routes:
            - id: svc-catch-all
              uri: no://op
              predicates:
                - Path=/svc/**
              filters:
                - SetStatus=BAD_GATEWAY
            - id: ws
              uri: lb:ws://sse
              predicates:
                - Path=/ws/sse/**
              filters:
                - StripPrefix=2
            - id: ui
              uri: lb://management-ui
              predicates:
                - Path=/**

The only special thing might be that my gateway is on ouath2 client and has a custom DelegatingServiceInstanceListSupplier but this should not be the cause of the exception, I guess...

[spencergibb]

If you can provide a complete, minimal, verifiable sample that is only gateway and no other dependencies (something that we can unzip attached to this issue or git clone, build, and deploy) that reproduces the problem, we can take a look.

[spring-cloud-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-cloud-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.