Config Server Does Not Return A PropertySource For Each Config Map or Secret

[susimsek]

Hi, Can you add config client integration documentation for Spring Cloud Kubernetes Config Server.

I have 2 questions that I'm curious about this topic. Can you share code snippets related to these questions? thank you in advance:)

1.How can I fetch property values for different profiles on Spring Cloud Kubernetes Config Server?

2. How can I define property values for different profiles on Spring Cloud Kubernetes Config Server?

[ryanjbaxter]

Spring Cloud Kubernetes Config Server is just a Spring Cloud Config Server with an optional environment repository for Config Maps and Secrets. Fetching property values and separating property values is done just like any other config server.

https://docs.spring.io/spring-cloud-config/docs/4.0.4/reference/html

[susimsek]

i created sample configmap as follows for default,qa and prod profiles. i can access the prod or qa property values on the kubernetes config server but my app cannot access the prod and kubernetes property files on the config server.

I'm getting an error like following. database connection information is defined in the account k8s profile on the config server. But my app cannot access any properties on the config server for k8s and prod profiles.

Can you share config client sample for Spring Cloud Kubernetes Config Server? The reproduce example codes are shared via my Github, check here.
 Spring Cloud Config Server version: 3.0.4

App startup log

App error log

App k8s config on Spring Cloud Kubernetes Config Server

App prod config on Spring Cloud Kubernetes Config Server

App aplication.yml

spring:
  application:
    name: account
  profiles:
    active: k8s, prod
  config:
    import: configserver:http://localhost:8888/

App configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: account
  namespace: default
data:
  application.yml: |-
    build:
      version: "3.0"

    account:
      message: "Welcome to EazyBank account related local APIs "
      contactDetails:
        name: "John Doe - Developer"
        email: "john@eazybank.com"
      onCallSupport:
        - (555) 555-1234
        - (555) 523-1345

    spring:
      main:
        allow-bean-definition-overriding: true
      output:
        ansi:
          enabled: always

    management:
      endpoints:
        web:
          exposure:
            include: "*"
      health:
        readiness-state:
          enabled: true
        liveness-state:
          enabled: true
      endpoint:
        shutdown:
          enabled: true
        health:
          probes:
            enabled: true
      info:
        env:
          enabled: true
      metrics:
        tags:
          application: ${spring.application.name}
      tracing:
        sampling:
          probability: 1.0
        enabled: true

    springdoc:
      oAuthFlow:
        tokenUrl: http://localhost:7080/realms/eazybank/protocol/openid-connect/token
    ---
    spring:
      config:
        activate:
          on-profile: qa
      main:
        allow-bean-definition-overriding: true
      output:
        ansi:
          enabled: always
    build:
      version: "2.0"

    account:
      message: "Welcome to EazyBank account related QA APIs "
      contactDetails:
        name: "Smitha Ray - QA Lead"
        email: "smitha@eazybank.com"
      onCallSupport:
        - (666) 265-3765
        - (666) 734-8371

    management:
      endpoints:
        web:
          exposure:
            include: "*"
      health:
        readiness-state:
          enabled: true
        liveness-state:
          enabled: true
      endpoint:
        shutdown:
          enabled: true
        health:
          probes:
            enabled: true
      info:
        env:
          enabled: true
      metrics:
        tags:
          application: ${spring.application.name}
      tracing:
        sampling:
          probability: 1.0
        enabled: true
    ---
    spring:
      config:
        activate:
          on-profile: prod
      main:
        allow-bean-definition-overriding: true
      output:
        ansi:
          enabled: always
      devtools:
        restart:
          enabled: false
        livereload:
          enabled: false
    server:
      shutdown: graceful

    build:
      version: "1.0"

    account:
      message: "Welcome to EazyBank account related production APIs "
      contactDetails:
        name: "Reine Aishwarya - Product Owner"
        email: "{cipher}acfc59b7eb19990a084cf8e17024511670254f67d8b765bd2fda3538f85db0cf81a698183045520fd5e381e1d4efea66"
      onCallSupport:
        - (453) 392-4829
        - (236) 203-0384

    management:
      endpoints:
        web:
          exposure:
            include: "*"
      health:
        readiness-state:
          enabled: true
        liveness-state:
          enabled: true
      endpoint:
        shutdown:
          enabled: true
        health:
          probes:
            enabled: true
      info:
        env:
          enabled: true
      metrics:
        tags:
          application: ${spring.application.name}
      tracing:
        sampling:
          probability: 1.0
        enabled: true
    ---
    spring:
      config:
        activate:
          on-profile: k8s
      datasource:
        url: jdbc:postgresql://postgresql:5432/accountdb
        username: easybank
        password: "{cipher}9e37167bf33b458463119a7927279b4f53ea984472903a0c2b61197a8739714b"
        hikari:
          maximum-pool-size: 30
          minimum-idle: 1
          pool-name: Hikari
          auto-commit: false
      jpa:
        hibernate:
          ddl-auto: none
          naming:
            physical-strategy: org.hibernate.boot.model.naming.CamelCaseToUnderscoresNamingStrategy
            implicit-strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
        show-sql: true
        open-in-view: false
        properties:
          hibernate.connection.provider_disables_autocommit: true
          hibernate.cache.redisson.fallback: true
          hibernate.cache.use_second_level_cache: true
          hibernate.cache.use_query_cache: true
          hibernate.generate_statistics: false
          hibernate.jdbc.batch_size: 25
          hibernate.order_inserts: true
          hibernate.order_updates: true
          hibernate.query.fail_on_pagination_over_collection_fetch: true
          hibernate.query.in_clause_parameter_padding: true
          jakarta.persistence.sharedCache.mode: ENABLE_SELECTIVE
          hibernate.cache.redisson.entity.expiration.max_entries: 10000
          hibernate.cache.redisson.entity.expiration.time_to_live: 3600000
          hibernate.cache.redisson.entity.expiration.max_idle_time: 1800000
          hibernate.cache.redisson.query.eviction.max_entries: 10000
          hibernate.cache.redisson.query.eviction.time_to_live: 3600000
          hibernate.cache.redisson.query.eviction.max_idle_time: 1800000
          org.hibernate.envers.audit_table_suffix: _history
          org.hibernate.envers.revision_field_name: revision
          org.hibernate.envers.revision_type_field_name: revision_type
      data:
        redis:
          host: redis-master
          port: 6379
          password: "{cipher}72f6a7d004cec8a7558dd0a94bd59b3b10205cfdad18d890d510e57ed17c5d7f44554bd293cdd529d1192202dad75c3d9c5ef970f16dab4e0c4d66f2df708b38"
      kafka:
        bootstrap-servers: http://kafka:9092
      cloud:
        kubernetes:
          discovery:
            all-namespaces: true
          discoveryServer:
            url: http://spring-cloud-kubernetes-discoveryserver:80/

    server:
      port: 8080

    eureka:
      client:
        enabled: false

    management:
      zipkin:
        tracing:
          endpoint: http://tempo-grafana-tempo-distributor:9411

[ryanjbaxter]

Your sample is way to complex to decipher or even reuse to reproduce the problem.

If you hit the /actuator/env endpoint when the k8s and prod profiles are active what is the result?

[susimsek]

i created simple project for config client integration. It seems that it only fetched the default profile on the spring cloud kubernetes config server . It did not fetch property values for prod and k8s environments. The reproduce example codes are shared via my Github, check here.

You can test the property values by typing the following command. Only default property values are loaded

curl --location 'http://localhost:8080/api/contact-info'

Actuator env api response

default profile on Spring Cloud Kubernetes Config Server

.

k8s profile on Spring Cloud Kubernetes Config Server

prod profile on Spring Cloud Kubernetes Config Server

[ryanjbaxter]

https://github.com/susimsek/spring-cloud-kubernetes-config-server-sample

This returns a 404

[susimsek]

i changed project visibility.it is currently public.

[ryanjbaxter]

Thanks this helps. There are actually 2 problems that this illustrates.

First the kubernetes config server will return property sources from the same config map with the same property source name. For example, when the config client requests configuration data without any profiles you will get a response like

{
  "name": "account",
  "profiles": [
    "default"
  ],
  "label": null,
  "version": null,
  "state": null,
  "propertySources": [
    {
      "name": "configmap.account.default",
      "source": {
        "build.version": "3.0",
        "account.message": "Welcome to EazyBank account related local APIs ",
        "account.contactDetails.email": "john@eazybank.com",
        "account.onCallSupport[1]": "(555) 523-1345",
        "account.onCallSupport[0]": "(555) 555-1234",
        "account.contactDetails.name": "John Doe - Developer"
      }
    }
  ]
}

Then the config client will make a second request with all active profiles (in you example k8s and prod) and it will return

{
  "name": "account",
  "profiles": [
    "k8s,prod"
  ],
  "label": null,
  "version": null,
  "state": null,
  "propertySources": [
    {
      "name": "configmap.account.default",
      "source": {
        "build.version": "1.0",
        "account.message": "Welcome to EazyBank account related kubernetes APIs ",
        "account.contactDetails.email": "k8ssmitha@eazybank.com",
        "account.onCallSupport[1]": "(236) 203-0384",
        "account.onCallSupport[0]": "(453) 392-4829",
        "spring.config.activate.on-profile": "k8s",
        "account.contactDetails.name": "Reine Aishwarya - Product Owner"
      }
    }
  ]
}

Since Spring Boot is keeping track of all the property sources that we add to the environment and both property sources get added with the same name the one without any active profiles overrides the k8s and prod profile one. We can fix this by making the property source names unique (for example by appending any active profiles).

Second problem relates the the response from the config server when there are active profiles.

What we should be returning is all the property sources for all active profiles, so in your example I would expect to see the prod and k8s and default property sources returned and then the client will resolve which one takes priority based on the order they are returned.

This is going to take some digging in order to figure out what the right solution is.

[wind57]

just FYI, next year I plan to start digging into config-server a lot more and potentially will reach this bug also

[ryanjbaxter]

I am going to dig into this, I am already down the rabbit hole

[Mean-Machine-Dee]

@susimsek in the ConfigMap why did you use cipher to encrypt the password for database connection and email. My thinking is, shouldn't this be stored in secrets. And if am right how will you load the secrets to the final response when you call the config server client profile. eg http://localhost:8080/account/prod