search

spring-projects / spring-boot

Spring Boot helps you to create Spring-powered, production-grade applications and services with absolute minimum fuss.
https://spring.io/projects/spring-boot
Apache License 2.0
75.22k stars 40.7k forks source link

Spring Boot using cloud config client pulling Multi-document YAML leaks the last profile properties into runtime configuration #33946

Closed jaschenk closed 1 year ago

jaschenk commented 1 year ago

In Spring Boot using cloud config client pulling Multi-document YAML leaks the last profile properties into runtime configuration, from either config server or on classpath.

I originally opened this under spring-cloud-config for client, but reviewer suggested Spring Boot should be where this issue should be created. https://github.com/spring-cloud/spring-cloud-config/issues/2215

Migrating to Spring Boot 3.0.1 and using Spring-cloud-config-client version 4.0.0, it appears there has been some regression in processing the configuration files, either from classpath or from config server.

In a multi-document YAML configuration file on the configuration server and initial 'application.yml' file specifies spring.config.import to initiate the configuration to be pulled.

Example

xyz-app-name.yml on config server contains:
server:
  port: 8080

spring:
  application:
    name: xyz-app-name
---
spring:  
 config:
    activate:
       profile: dev
#
# External Services
nve:
 resources:
    external:
          svc:
            endpoint:
                url: https://api2-dev.xyz.com/someendpoint 
---
spring:  
 config:
    activate:
       profile: tst
#
# External Services
nve:
 resources:
    external:
          svc:
            endpoint:
                url: https://api2-tst.xyz.com/someendpoint                     
---
spring:
  config:
    activate:
       profile: prd
#
# External Services
nve:
 resources:
    external:
          svc:
            endpoint:
                url: https://api2.xyz.com/someendpoint  
#
---

When specify the spring.active.profles=dev, we see 'dev' profile being used, but after a short time the 'prd' property values are injected!

We should see ' https://api2-dev.xyz.com/someendpoint' as our dev uri, instead it is populating with the 'prd' value: 'https://api2.xyz.com/someendpoint'.

This appears to be also related to Issue # 1761: https://github.com/spring-cloud/spring-cloud-config/issues/1761 .

Dependency Tree:
[INFO] 
[INFO] --- maven-dependency-plugin:3.3.0:tree (default-cli) @ ddp-cissvc ---
[INFO] io.bh.bhe.nve.dsp.datapower.impl:ddp-cissvc:jar:DEV-SNAPSHOT
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.19.0:compile
[INFO] |  \- org.apache.logging.log4j:log4j-api:jar:2.19.0:compile
[INFO] +- org.slf4j:slf4j-simple:jar:1.7.36:compile
[INFO] |  \- org.slf4j:slf4j-api:jar:2.0.6:compile

--REDACTED--

[INFO] |  +- org.springframework.boot:spring-boot-starter-web-services:jar:3.0.1:compile
[INFO] |  |  +- jakarta.xml.ws:jakarta.xml.ws-api:jar:4.0.0:compile
[INFO] |  |  \- org.springframework:spring-oxm:jar:6.0.3:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-web:jar:3.0.1:compile
[INFO] |  |  +- org.springframework.boot:spring-boot-starter-json:jar:3.0.1:compile
[INFO] |  |  +- org.springframework.boot:spring-boot-starter-tomcat:jar:3.0.1:compile
[INFO] |  |  |  +- org.apache.tomcat.embed:tomcat-embed-core:jar:10.1.4:compile
[INFO] |  |  |  +- org.apache.tomcat.embed:tomcat-embed-el:jar:10.1.4:compile
[INFO] |  |  |  \- org.apache.tomcat.embed:tomcat-embed-websocket:jar:10.1.4:compile
[INFO] |  |  \- org.springframework:spring-webmvc:jar:6.0.3:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-cache:jar:3.0.1:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-thymeleaf:jar:3.0.1:compile
[INFO] |  |  \- org.thymeleaf:thymeleaf-spring6:jar:3.1.1.RELEASE:compile
[INFO] |  |     \- org.thymeleaf:thymeleaf:jar:3.1.1.RELEASE:compile
[INFO] |  |        +- org.attoparser:attoparser:jar:2.0.6.RELEASE:compile
[INFO] |  |        \- org.unbescape:unbescape:jar:1.1.6.RELEASE:compile
[INFO] |  +- org.webjars:bootstrap:jar:5.2.0:compile
[INFO] |  +- org.webjars:jquery:jar:3.6.1:compile
[INFO] |  +- org.webjars:font-awesome:jar:6.2.0:compile
[INFO] |  +- org.webjars:webjars-locator:jar:0.45:compile
[INFO] |  |  +- org.webjars:webjars-locator-core:jar:0.52:compile
[INFO] |  |  |  \- io.github.classgraph:classgraph:jar:4.8.147:compile
[INFO] |  |  \- com.github.spullara.mustache.java:compiler:jar:0.9.10:compile
[INFO] |  +- org.json:json:jar:20220924:compile
[INFO] |  +- org.jdom:jdom:jar:2.0.2:compile
[INFO] |  +- commons-io:commons-io:jar:2.8.0:compile
[INFO] |  +- org.aspectj:aspectjweaver:jar:1.9.19:compile
[INFO] |  +- org.apache.commons:commons-lang3:jar:3.12.0:compile
[INFO] |  +- org.apache.commons:commons-text:jar:1.10.0:compile
[INFO] |  +- net.sf.saxon:Saxon-EE:jar:10.1:compile
[INFO] |  +- com.google.guava:guava:jar:31.1-jre:compile
[INFO] |  |  +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO] |  |  +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] |  |  +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] |  |  +- org.checkerframework:checker-qual:jar:3.12.0:compile
[INFO] |  |  +- com.google.errorprone:error_prone_annotations:jar:2.11.0:compile
[INFO] |  |  \- com.google.j2objc:j2objc-annotations:jar:1.3:compile
[INFO] |  +- joda-time:joda-time:jar:2.11.1:compile
[INFO] |  \- org.jsoup:jsoup:jar:1.15.3:compile
[INFO] +- org.yaml:snakeyaml:jar:1.33:compile
[INFO] +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.14.1:compile
[INFO] +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.14.1:compile
[INFO] +- com.fasterxml.jackson.module:jackson-module-parameter-names:jar:2.14.1:compile
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.14.1:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.14.1:compile
[INFO] +- com.fasterxml.jackson.dataformat:jackson-dataformat-xml:jar:2.14.1:compile
[INFO] |  +- org.codehaus.woodstox:stax2-api:jar:4.2.1:compile
[INFO] |  \- com.fasterxml.woodstox:woodstox-core:jar:6.4.0:compile
[INFO] +- jakarta.xml.bind:jakarta.xml.bind-api:jar:4.0.0:compile
[INFO] |  \- jakarta.activation:jakarta.activation-api:jar:2.1.0:compile
[INFO] +- com.sun.xml.bind:jaxb-impl:jar:4.0.1:compile
[INFO] |  \- com.sun.xml.bind:jaxb-core:jar:4.0.1:compile
[INFO] |     \- org.eclipse.angus:angus-activation:jar:1.0.0:runtime

--REDACTED--

[INFO] +- org.springframework.boot:spring-boot-starter-test:jar:3.0.1:test
[INFO] |  +- org.springframework.boot:spring-boot-test:jar:3.0.1:test
[INFO] |  +- org.springframework.boot:spring-boot-test-autoconfigure:jar:3.0.1:test
[INFO] |  +- com.jayway.jsonpath:json-path:jar:2.7.0:test
[INFO] |  |  \- net.minidev:json-smart:jar:2.4.8:test
[INFO] |  |     \- net.minidev:accessors-smart:jar:2.4.8:test
[INFO] |  |        \- org.ow2.asm:asm:jar:9.1:test
[INFO] |  +- org.hamcrest:hamcrest:jar:2.2:test
[INFO] |  +- org.junit.jupiter:junit-jupiter:jar:5.9.1:test
[INFO] |  |  +- org.junit.jupiter:junit-jupiter-params:jar:5.9.1:test
[INFO] |  |  \- org.junit.jupiter:junit-jupiter-engine:jar:5.9.1:test
[INFO] |  |     \- org.junit.platform:junit-platform-engine:jar:1.9.1:test
[INFO] |  +- org.skyscreamer:jsonassert:jar:1.5.1:test
[INFO] |  +- org.springframework:spring-core:jar:6.0.3:compile
[INFO] |  |  \- org.springframework:spring-jcl:jar:6.0.3:compile
[INFO] |  \- org.springframework:spring-test:jar:6.0.3:test
[INFO] +- org.springframework.ws:spring-ws-test:jar:4.0.0:test
[INFO] |  +- org.springframework.ws:spring-xml:jar:4.0.0:compile
[INFO] |  +- org.springframework.ws:spring-ws-core:jar:4.0.0:compile
[INFO] |  |  \- jakarta.xml.soap:jakarta.xml.soap-api:jar:3.0.0:compile
[INFO] |  +- org.springframework:spring-context:jar:6.0.3:compile
[INFO] |  |  \- org.springframework:spring-expression:jar:6.0.3:compile
[INFO] |  +- xmlunit:xmlunit:jar:1.6:test
[INFO] |  +- com.sun.xml.messaging.saaj:saaj-impl:jar:3.0.0:compile
[INFO] |  \- org.jvnet.staxex:stax-ex:jar:2.1.0:compile
[INFO] +- org.assertj:assertj-core:jar:3.23.1:test
[INFO] |  \- net.bytebuddy:byte-buddy:jar:1.12.20:test
[INFO] +- org.xmlunit:xmlunit-core:jar:2.9.0:test
[INFO] +- org.xmlunit:xmlunit-assertj3:jar:2.9.0:test
[INFO] +- org.mockito:mockito-core:jar:4.6.1:test
[INFO] |  +- net.bytebuddy:byte-buddy-agent:jar:1.12.20:test
[INFO] |  \- org.objenesis:objenesis:jar:3.2:test
[INFO] +- org.mockito:mockito-inline:jar:4.6.1:test
[INFO] +- org.springframework.boot:spring-boot-starter:jar:3.0.1:compile
[INFO] |  +- org.springframework.boot:spring-boot:jar:3.0.1:compile
[INFO] |  +- org.springframework.boot:spring-boot-autoconfigure:jar:3.0.1:compile
[INFO] |  +- org.springframework.boot:spring-boot-starter-logging:jar:3.0.1:compile
[INFO] |  |  \- org.slf4j:jul-to-slf4j:jar:2.0.6:compile
[INFO] |  \- jakarta.annotation:jakarta.annotation-api:jar:2.1.1:compile
[INFO] +- org.springframework.boot:spring-boot-starter-log4j2:jar:3.0.1:compile
[INFO] |  \- org.apache.logging.log4j:log4j-slf4j2-impl:jar:2.19.0:compile
[INFO] +- org.springframework:spring-context-support:jar:6.0.3:compile
[INFO] |  \- org.springframework:spring-beans:jar:6.0.3:compile
[INFO] +- org.springframework.boot:spring-boot-starter-actuator:jar:3.0.1:compile
[INFO] |  +- org.springframework.boot:spring-boot-actuator-autoconfigure:jar:3.0.1:compile
[INFO] |  |  \- org.springframework.boot:spring-boot-actuator:jar:3.0.1:compile
[INFO] |  \- io.micrometer:micrometer-observation:jar:1.10.2:compile
[INFO] +- org.springframework.boot:spring-boot-starter-aop:jar:3.0.1:compile
[INFO] |  \- org.springframework:spring-aop:jar:6.0.3:compile
[INFO] +- org.springframework.retry:spring-retry:jar:2.0.0:compile
[INFO] +- org.springframework:spring-aspects:jar:6.0.3:compile
[INFO] +- org.springframework.cloud:spring-cloud-config-client:jar:4.0.0:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-commons:jar:4.0.0:compile
[INFO] |  |  \- org.springframework.security:spring-security-crypto:jar:6.0.1:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-context:jar:4.0.0:compile
[INFO] |  +- org.springframework:spring-web:jar:6.0.3:compile
[INFO] |  \- org.apache.httpcomponents.client5:httpclient5:jar:5.1.4:compile
[INFO] |     +- org.apache.httpcomponents.core5:httpcore5:jar:5.1.5:compile
[INFO] |     +- org.apache.httpcomponents.core5:httpcore5-h2:jar:5.1.5:compile
[INFO] |     \- commons-codec:commons-codec:jar:1.15:compile
[INFO] +- org.springframework.cloud:spring-cloud-starter-consul-discovery:jar:4.0.0:compile
[INFO] |  +- org.springframework.cloud:spring-cloud-starter-consul:jar:4.0.0:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-starter:jar:4.0.0:compile
[INFO] |  |  |  \- org.springframework.security:spring-security-rsa:jar:1.0.11.RELEASE:compile
[INFO] |  |  |     \- org.bouncycastle:bcpkix-jdk15on:jar:1.69:compile
[INFO] |  |  |        +- org.bouncycastle:bcprov-jdk15on:jar:1.69:compile
[INFO] |  |  |        \- org.bouncycastle:bcutil-jdk15on:jar:1.69:compile
[INFO] |  |  +- org.springframework.cloud:spring-cloud-consul-core:jar:4.0.0:compile
[INFO] |  |  |  \- org.springframework.boot:spring-boot-starter-validation:jar:3.0.1:compile
[INFO] |  |  |     \- org.hibernate.validator:hibernate-validator:jar:8.0.0.Final:compile
[INFO] |  |  |        +- jakarta.validation:jakarta.validation-api:jar:3.0.2:compile
[INFO] |  |  |        +- org.jboss.logging:jboss-logging:jar:3.5.0.Final:compile
[INFO] |  |  |        \- com.fasterxml:classmate:jar:1.5.1:compile
[INFO] |  |  +- com.ecwid.consul:consul-api:jar:1.4.5:compile
[INFO] |  |  +- com.google.code.gson:gson:jar:2.9.1:compile
[INFO] |  |  +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile
[INFO] |  |  \- org.apache.httpcomponents:httpcore:jar:4.4.16:compile
[INFO] |  \- org.springframework.cloud:spring-cloud-consul-discovery:jar:4.0.0:compile
[INFO] +- io.micrometer:micrometer-core:jar:1.10.2:compile
[INFO] |  +- io.micrometer:micrometer-commons:jar:1.10.2:compile
[INFO] |  +- org.hdrhistogram:HdrHistogram:jar:2.1.12:runtime
[INFO] |  \- org.latencyutils:LatencyUtils:jar:2.0.3:runtime
[INFO] +- io.micrometer:micrometer-registry-jmx:jar:1.10.2:compile
[INFO] |  \- io.dropwizard.metrics:metrics-jmx:jar:4.2.14:compile
[INFO] |     \- io.dropwizard.metrics:metrics-core:jar:4.2.14:compile
[INFO] +- io.micrometer:micrometer-registry-prometheus:jar:1.10.2:compile
[INFO] |  \- io.prometheus:simpleclient_common:jar:0.16.0:compile
[INFO] +- io.prometheus:simpleclient_pushgateway:jar:0.16.0:compile
[INFO] |  \- io.prometheus:simpleclient:jar:0.16.0:compile
[INFO] |     +- io.prometheus:simpleclient_tracer_otel:jar:0.16.0:compile
[INFO] |     |  \- io.prometheus:simpleclient_tracer_common:jar:0.16.0:compile
[INFO] |     \- io.prometheus:simpleclient_tracer_otel_agent:jar:0.16.0:compile
[INFO] +- io.micrometer:micrometer-tracing:jar:1.0.0:compile
[INFO] |  +- io.micrometer:context-propagation:jar:1.0.0:compile
[INFO] |  \- aopalliance:aopalliance:jar:1.0:compile
[INFO] +- io.pivotal.cfenv:java-cfenv:jar:2.4.1:runtime
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.14.1:compile
[INFO] +- org.projectlombok:lombok:jar:1.18.24:provided
[INFO] \- org.mockito:mockito-junit-jupiter:jar:4.8.0:test
[INFO]    \- org.junit.jupiter:junit-jupiter-api:jar:5.9.1:test
[INFO]       +- org.opentest4j:opentest4j:jar:1.2.0:test
[INFO]       +- org.junit.platform:junit-platform-commons:jar:1.9.1:test
[INFO]       \- org.apiguardian:apiguardian-api:jar:1.1.2:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  7.802 s
[INFO] Finished at: 2023-01-20T18:33:32-08:00
[INFO] ------------------------------------------------------------------------

Please advise on what additional information you may need to further debug.

Many thanks, Jeff Schenk

wilkinsona commented 1 year ago

You said in the Spring Cloud issue that the problem occurs with a single multi-document YAML file and without involving Spring Cloud Config. Can you please provide a minimal example of that?

jaschenk commented 1 year ago

Hello, attached in a zip of a redacted application.yml file. Which is a Multi-Document YAML file. If I reference this using 'dev' profile, I see 'prd' leaking into property values.

If I eliminate all other profiles below the 'dev' profile, the service boots with the correct property values. application.zip

Also, I tried using a property file generated from my YAML using the new #--- profile separators and that failed as well. And if for some reason you end the Multi-Document yaml using a '---' separator other weirdness occurs.

Please advise if you need any additional information.

Many Thanks, Jeff

philwebb commented 1 year ago

@jaschenk Thanks for the YAML file, but we really need a complete application that we can run and debug to diagnose the problem. Are you able to provide a minimal reproducible example application as either a GitHub project or attached to this issue as a zip. Ideally the sample won't include any Spring Cloud dependencies.

jaschenk commented 1 year ago

@philwebb Put together simple demo, but unable to reproduce in a simple demo. It must be something with the inclusion of config server in some way or other dependency in stack, but not sure. Will keep investigating on my side to see if I can get additional insight. Thanks, Jeff

jaschenk commented 1 year ago

Looks like issue is in fact related to cloud config. Used a work around of creating distinct application-.yml YAML files instead of Multi-Document YAML. Closing.