spring-cloud / spring-cloud-consul

Spring Cloud Consul
http://cloud.spring.io/spring-cloud-consul/
Apache License 2.0
805 stars 543 forks source link

Error getting config from consul if two profiles specified with different values of some properties #802

Open TimurDaas opened 1 year ago

TimurDaas commented 1 year ago

I've updated spring cloud from 2021.0.4 to 2021.0.5. After updating I get an error while starting my spring-boot application. I noticed, that problem is because I have test profile with disabled cloud services. If I comment all properties which has **.discovery.** in name - it works Of course it doesn't work in 'test' profile, because by default, discovery is enabled.

As I understand, such behaviour happens because of https://github.com/spring-cloud/spring-cloud-consul/issues/785 Is that bug? Or I have mistakes in my configuration?

my application.yml

server:
  port: 8100

spring:
  application:
    name: rabbitmq-client
  config:
    import: "configserver:,consul:"

  cloud:
    stream:
      bindings:
        test:
          destination: data_broker
          group: client1_group
        process-in-0:
          destination: data_broker
          group: client_group
    consul:
      config:
        enabled: true
        format: YAML
        prefixes: config
        default-context: rabbitmq-client
        data-key: data
        profile-separator: '::'
        name: rabbitmq-client
      discovery:
        instance-id: ${spring.application.name}
        prefer-ip-address: true
        metadata:
          version: 1.0
      host: ${CLOUD_SERVICES_ADDR}
      port: 8500

    config:
      enabled: true
      discovery:
        enabled: true
        service-id: config-service

  rabbitmq:
    host: ${CLOUD_SERVICES_ADDR}
    port: 5672
    username: ${RABBITMQ_USER:}
    password: ${RABBITMQ_PASSWORD:}

---
spring:
  config:
    activate:
      on-profile: test

  cloud:
    consul:
      discovery:
        enabled: false
      enabled: false
      config:
        enabled: false
    discovery:
      enabled: false
    config:
      enabled: false
      discovery:
        enabled: false

Stacktrace:

02:21:30.899 [restartedMain] DEBUG org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter - Application failed to start due to an exception
org.springframework.boot.context.config.ConfigDataResourceNotFoundException: Config data resource '[ConsulConfigDataResource@651d884b context = 'config/rabbitmq-client::default/', optional = true, properties = [ConsulConfigProperties@27342471 enabled = true, prefixes = list['config'], defaultContext = 'rabbitmq-client', profileSeparator = '::', format = YAML, dataKey = 'data', aclToken = [null], watch = [ConsulConfigProperties.Watch@59f16715 waitTime = 55, enabled = true, delay = 1000], failFast = true, name = 'rabbitmq-client'], profile = 'default']' via location 'consul:' cannot be found
    at org.springframework.boot.context.config.ConfigDataResourceNotFoundException.withLocation(ConfigDataResourceNotFoundException.java:97)
    at org.springframework.boot.context.config.ConfigDataImporter.handle(ConfigDataImporter.java:145)
    at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:136)
    at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
    at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:116)
    at org.springframework.boot.context.config.ConfigDataEnvironment.processWithProfiles(ConfigDataEnvironment.java:311)
    at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:232)
    at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:102)
    at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:94)
    at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:102)
    at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:87)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
    at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:85)
    at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:66)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:120)
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:114)
    at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:65)
    at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:343)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:301)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
    at ua.com.timur.rabbitmqclient.RabbitmqClientApplication.main(RabbitmqClientApplication.java:12)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'spring.cloud.config.discovery.enabled' to java.lang.Boolean
    at org.springframework.boot.context.properties.bind.Binder.handleBindError(Binder.java:387)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:347)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:332)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:262)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:223)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:210)
    at org.springframework.cloud.consul.discovery.configclient.ConsulConfigServerBootstrapper.isDiscoveryEnabled(ConsulConfigServerBootstrapper.java:102)
    at org.springframework.cloud.consul.discovery.configclient.ConsulConfigServerBootstrapper.lambda$initialize$1(ConsulConfigServerBootstrapper.java:58)
    at org.springframework.boot.DefaultBootstrapContext.getInstance(DefaultBootstrapContext.java:119)
    at org.springframework.boot.DefaultBootstrapContext.getOrElseThrow(DefaultBootstrapContext.java:111)
    at org.springframework.boot.DefaultBootstrapContext.get(DefaultBootstrapContext.java:88)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.getBean(ConsulConfigDataLoader.java:103)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.doLoad(ConsulConfigDataLoader.java:62)
    at org.springframework.cloud.consul.config.ConsulRetryBootstrapper.lambda$null$2(ConsulRetryBootstrapper.java:61)
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:209)
    at org.springframework.cloud.consul.config.ConsulRetryBootstrapper.lambda$null$3(ConsulRetryBootstrapper.java:60)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.load(ConsulConfigDataLoader.java:54)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.load(ConsulConfigDataLoader.java:38)
    at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:107)
    at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:128)
    ... 28 common frames omitted
Caused by: org.springframework.boot.context.config.InactiveConfigDataAccessException: Inactive property source 'Config resource 'class path resource [application.yml]' via location 'optional:classpath:/' (document #1)' imported from location 'class path resource [application.yml]' cannot contain property 'spring.cloud.config.discovery.enabled' [origin: class path resource [application.yml] - 72:18]
    at org.springframework.boot.context.config.InactiveConfigDataAccessException.throwIfPropertyFound(InactiveConfigDataAccessException.java:126)
    at org.springframework.boot.context.config.ConfigDataEnvironmentContributors$InactiveSourceChecker.onSuccess(ConfigDataEnvironmentContributors.java:310)
    at org.springframework.boot.context.properties.bind.Binder.handleBindResult(Binder.java:354)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:344)
    ... 47 common frames omitted
02:21:30.899 [restartedMain] ERROR org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter - 

***************************
APPLICATION FAILED TO START
***************************

Description:

Config data resource '[ConsulConfigDataResource@651d884b context = 'config/rabbitmq-client::default/', optional = true, properties = [ConsulConfigProperties@27342471 enabled = true, prefixes = list['config'], defaultContext = 'rabbitmq-client', profileSeparator = '::', format = YAML, dataKey = 'data', aclToken = [null], watch = [ConsulConfigProperties.Watch@59f16715 waitTime = 55, enabled = true, delay = 1000], failFast = true, name = 'rabbitmq-client'], profile = 'default']' via location 'consul:' does not exist

Action:

Check that the value 'consul:' at class path resource [application.yml] - 10:13 is correct, or prefix it with 'optional:'
spencergibb commented 1 year ago

spring.config.import is always evaluated regardless of profiles. So the values in the test profile have no effect.

spring-cloud-issues commented 1 year ago

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

TimurDaas commented 1 year ago

@spencergibb But why this doen't work even in default profile? When I want to enable config from discovery. In this case, I expect that properties defined under test profile will be ignored. But in reality I got an error like above.

Maybe, I don't clear understand some details. Can you advice what I need to chage in my config, if I want to enable two config sources(from config-server located via consul-discovery and from consul key-value) by default, and to disable config and discovery during tests? I will really appreciate it if you point me in the right direction! Thanks!

TimurDaas commented 1 year ago

I have also tried to put spring.config.import and all props that enable cloud-config into profile-specified block. In this case it only helps me to run application under test profile without errors. But in cloud profile(created new profile specially for spring.config.import) it stills fails starting.

New application.yml:

server:
  port: 8100

spring:
  profiles:
    active: cloud
  application:
    name: rabbitmq-client

---
spring:
  config:
    activate:
      on-profile: cloud

    import: "configserver:,consul:"

  cloud:
    stream:
      bindings:
        test:
          destination: data_broker
          group: client1_group
        process-in-0:
          destination: data_broker
          group: client_group
    consul:
      config:
        enabled: true
        format: YAML
        prefixes: config
        default-context: rabbitmq-client
        data-key: data
        profile-separator: '::'
        name: rabbitmq-client
      discovery:
        instance-id: ${spring.application.name}
        prefer-ip-address: true
        metadata:
          version: 1.0
      host: ${IFOBS_CLOUD_SERVICES_ADDR}
      port: 8500

    config:
      enabled: true
      discovery:
        enabled: true
        service-id: config-service

  rabbitmq:
    host: ${IFOBS_CLOUD_SERVICES_ADDR}
    port: 5672
    username: ${RABBITMQ_USER:}
    password: ${RABBITMQ_PASSWORD:}

---
spring:
  config:
    activate:
      on-profile: test

  cloud:
    consul:
      discovery:
        enabled: false
      enabled: false
      config:
        enabled: false
    discovery:
      enabled: false
    config:
      enabled: false
      discovery:
        enabled: false

Error:

01:49:22.436 [Thread-0] DEBUG org.springframework.boot.devtools.restart.classloader.RestartClassLoader - Created RestartClassLoader org.springframework.boot.devtools.restart.classloader.RestartClassLoader@3585e46
01:49:28.860 [restartedMain] DEBUG org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter - Application failed to start due to an exception
org.springframework.boot.context.config.ConfigDataResourceNotFoundException: Config data resource '[ConsulConfigDataResource@56cef68f context = 'config/rabbitmq-client::cloud/', optional = true, properties = [ConsulConfigProperties@2db3bd65 enabled = true, prefixes = list['config'], defaultContext = 'rabbitmq-client', profileSeparator = '::', format = YAML, dataKey = 'data', aclToken = [null], watch = [ConsulConfigProperties.Watch@17f2b8d0 waitTime = 55, enabled = true, delay = 1000], failFast = true, name = 'rabbitmq-client'], profile = 'cloud']' via location 'consul:' cannot be found
    at org.springframework.boot.context.config.ConfigDataResourceNotFoundException.withLocation(ConfigDataResourceNotFoundException.java:97)
    at org.springframework.boot.context.config.ConfigDataImporter.handle(ConfigDataImporter.java:145)
    at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:136)
    at org.springframework.boot.context.config.ConfigDataImporter.resolveAndLoad(ConfigDataImporter.java:86)
    at org.springframework.boot.context.config.ConfigDataEnvironmentContributors.withProcessedImports(ConfigDataEnvironmentContributors.java:116)
    at org.springframework.boot.context.config.ConfigDataEnvironment.processWithProfiles(ConfigDataEnvironment.java:311)
    at org.springframework.boot.context.config.ConfigDataEnvironment.processAndApply(ConfigDataEnvironment.java:232)
    at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:102)
    at org.springframework.boot.context.config.ConfigDataEnvironmentPostProcessor.postProcessEnvironment(ConfigDataEnvironmentPostProcessor.java:94)
    at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEnvironmentPreparedEvent(EnvironmentPostProcessorApplicationListener.java:102)
    at org.springframework.boot.env.EnvironmentPostProcessorApplicationListener.onApplicationEvent(EnvironmentPostProcessorApplicationListener.java:87)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143)
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:131)
    at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListener.java:85)
    at org.springframework.boot.SpringApplicationRunListeners.lambda$environmentPrepared$2(SpringApplicationRunListeners.java:66)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:120)
    at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:114)
    at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java:65)
    at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:343)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:301)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1303)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1292)
    at ua.com.timur.rabbitmqclient.RabbitmqClientApplication.main(RabbitmqClientApplication.java:12)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: org.springframework.boot.context.properties.bind.BindException: Failed to bind properties under 'spring.cloud.config.discovery.enabled' to java.lang.Boolean
    at org.springframework.boot.context.properties.bind.Binder.handleBindError(Binder.java:387)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:347)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:332)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:262)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:223)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:210)
    at org.springframework.cloud.consul.discovery.configclient.ConsulConfigServerBootstrapper.isDiscoveryEnabled(ConsulConfigServerBootstrapper.java:102)
    at org.springframework.cloud.consul.discovery.configclient.ConsulConfigServerBootstrapper.lambda$initialize$1(ConsulConfigServerBootstrapper.java:58)
    at org.springframework.boot.DefaultBootstrapContext.getInstance(DefaultBootstrapContext.java:119)
    at org.springframework.boot.DefaultBootstrapContext.getOrElseThrow(DefaultBootstrapContext.java:111)
    at org.springframework.boot.DefaultBootstrapContext.get(DefaultBootstrapContext.java:88)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.getBean(ConsulConfigDataLoader.java:103)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.doLoad(ConsulConfigDataLoader.java:62)
    at org.springframework.cloud.consul.config.ConsulRetryBootstrapper.lambda$null$2(ConsulRetryBootstrapper.java:61)
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:209)
    at org.springframework.cloud.consul.config.ConsulRetryBootstrapper.lambda$null$3(ConsulRetryBootstrapper.java:60)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.load(ConsulConfigDataLoader.java:54)
    at org.springframework.cloud.consul.config.ConsulConfigDataLoader.load(ConsulConfigDataLoader.java:38)
    at org.springframework.boot.context.config.ConfigDataLoaders.load(ConfigDataLoaders.java:107)
    at org.springframework.boot.context.config.ConfigDataImporter.load(ConfigDataImporter.java:128)
    ... 28 common frames omitted
Caused by: org.springframework.boot.context.config.InactiveConfigDataAccessException: Inactive property source 'Config resource 'class path resource [application.yml]' via location 'optional:classpath:/' (document #2)' imported from location 'class path resource [application.yml]' cannot contain property 'spring.cloud.config.discovery.enabled' [origin: class path resource [application.yml] - 75:18]
    at org.springframework.boot.context.config.InactiveConfigDataAccessException.throwIfPropertyFound(InactiveConfigDataAccessException.java:126)
    at org.springframework.boot.context.config.ConfigDataEnvironmentContributors$InactiveSourceChecker.onSuccess(ConfigDataEnvironmentContributors.java:310)
    at org.springframework.boot.context.properties.bind.Binder.handleBindResult(Binder.java:354)
    at org.springframework.boot.context.properties.bind.Binder.bind(Binder.java:344)
    ... 47 common frames omitted
01:49:28.861 [restartedMain] ERROR org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter - 

***************************
APPLICATION FAILED TO START
***************************

Description:

Config data resource '[ConsulConfigDataResource@56cef68f context = 'config/rabbitmq-client::cloud/', optional = true, properties = [ConsulConfigProperties@2db3bd65 enabled = true, prefixes = list['config'], defaultContext = 'rabbitmq-client', profileSeparator = '::', format = YAML, dataKey = 'data', aclToken = [null], watch = [ConsulConfigProperties.Watch@17f2b8d0 waitTime = 55, enabled = true, delay = 1000], failFast = true, name = 'rabbitmq-client'], profile = 'cloud']' via location 'consul:' does not exist

Action:

Check that the value 'consul:' at class path resource [application.yml] - 16:13 is correct, or prefix it with 'optional:'
TimurDaas commented 1 year ago

@spencergibb could you provide feedback, please?

spencergibb commented 1 year ago

I don't know why it is trying to get a value from an inactive resource

Caused by: org.springframework.boot.context.config.InactiveConfigDataAccessException: Inactive property source 'Config resource 'class path resource [application.yml]' via location 'optional:classpath:/' (document #1)' imported from location 'class path resource [application.yml]' cannot contain property 'spring.cloud.config.discovery.enabled' [origin: class path resource [application.yml] - 72:18]

I'll have to work with someone from the boot team. /cc @philwebb

TimurDaas commented 1 year ago

thanks will be waiting for your feedback parallelly I'll try to find either solution or bug in spring boot

spencergibb commented 1 year ago

Don't open anything in boot unless our investigation reveals that we need to

philwebb commented 1 year ago

I'm not 100% sure what's going on, but I suspect that the binder being used in this line is wrong or stale. Ideally, the binder should only be used in the ConfigDataLocationResolver.

@spencergibb Do you have a sample application for this one? Have you managed to reproduce it locally?

spencergibb commented 1 year ago

I have reproduced it. I can share tomorrow

TimurDaas commented 1 year ago

If you need my help with reproducing or fixing this bug, I would be glad to help you It will improve my professional skills

danikeichanka commented 11 months ago

Hello there,

Seems like I've faced this issue as well - if "spring.cloud.config.discovery" is enabled then I can't override Consul port in another profile as ConsulClient is initialized with default ConsulProperties values in that case. As the result - application can't connect to Consul as ConsulClient was initialized with invalid port.

Looks like issue is actually here - the Binder seems to be invalid as it doesn't bind properties from the current profile, only from initial "application.yaml" (while Binder in ConfigDataLocationResolverContext, for example, actually binds correct value).

I faced this issue using following versions:

There are application properties:

application.yaml

spring:
  cloud:
    consul:
      host: ${consul.host:localhost}
      discovery:
        prefer-ip-address: true
        health-check-timeout: 30s
        health-check-interval: 20s
        health-check-path: /actuator/health
        query-passing: true
    config:
      discovery:
        enabled: true
        service-id: config-service
      fail-fast: true
      enabled: true

application-dev.yaml

spring:
  config:
    import: "configserver:"
  cloud:
    consul:
      port: 9091
    config:
      discovery:
        enabled: true

So the ConsulProperties are initialized with default 8500 port instead of 9091 provided in "application-dev.yaml".

P.S. I would like to not use ConfigServer for some profiles (like 'local') this is why "spring.config.import" is not in "application.yaml", but in other profiles which will use ConfigServer