spring-cloud / spring-cloud-consul

Spring Cloud Consul
http://cloud.spring.io/spring-cloud-consul/
Apache License 2.0
805 stars 543 forks source link

Support multiple file names in data-key for YML format #835

Closed bpeterson3134 closed 3 months ago

bpeterson3134 commented 3 months ago

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] I'm trying to use spring-cloud consul (v4.0.4) to read properties from multiple yaml files with the same prefix. However data-key, only accepts a single file name.

Describe the solution you'd like A clear and concise description of what you want to happen. So say I have below config files I want to have consul manage and refresh using config watcher:

src/main/resources/application.yml
src/main/resources/application-dev.yml
src/main/resources/features/features.yml
src/main/resources/mappings/my-mappings-dev.yml

I want to provide a config like this:

spring:
  config:
    import: "optional:consul:"
  cloud:
    consul:
      discovery:
        enabled: ${features.config-client.${environment}.enabled}
      enabled: ${features.config-client.${environment}.enabled}
      scheme: https
      host: ${consul_host}
      port: 443
      config:
        fail-fast: true
        acl-token: ${consul_token}
        enabled: ${features.config-client.${environment}.enabled}
        default-context: dev
        prefixes:
          - my/custom/path
        format: YAML
        data-key: application.yml,application-dev.yml,features.yml,my-mappings-dev.yml 
        watch:
          enabled: ${features.config-client.${environment}.enabled}
          delay: 300000
          wait-time: 5

Another option that would work for me is if I reference one file like application-dev.yml and use spring.config.import to import the other ones, then consul would refresh all 4 files even though data-key is set to application-dev.yml. However, Im thinking the first solution would maybe provide more flexibility?

Describe alternatives you've considered I've just been considering consul for externalizing multiple configs although it seems it only supports a single externalized config. Im unable to use different key prefixes for each file because the way things work from within our company is that a single github repository is onboarded and given a single prefix. Then I can onboard multiple files for that prefix/repository. Then anytime a file is changed in github and webhook is sent to consul server to update the properties in consul for the changes (now github and consul are in sync). Everything works fine for each individual file, but unable to get multiple files to be read and refreshed using consul. One of the reasons is for this snippet of code in ConsulPropertySource:

protected void parsePropertiesWithNonKeyValueFormat(List<GetValue> values, ConsulConfigProperties.Format format) {
        if (values != null) {
            Iterator var3 = values.iterator();

            while(var3.hasNext()) {
                GetValue getValue = (GetValue)var3.next();
                String key = getValue.getKey().replace(this.context, "");
                if (this.configProperties.getDataKey().equals(key)) {
                    this.parseValue(getValue, format);
                }
            }

        }
    }

Additional context Add any other context or screenshots about the feature request here.

bpeterson3134 commented 3 months ago

Finally figured this out for anyone trying to do similar...below is the config that worked. Change to FILES made it work. Also note the change in spring.config.import.


spring:
  config:
    import: "optional:consul:${consul_host}/my/custom/path/dev/application.yml;/my/custom/path/dev/application-dev.yml;/my/custom/path/dev/features.yml;/my/custom/path/dev/my-mappings-dev.yml"
  cloud:
    consul:
      discovery:
        enabled: ${features.config-client.${environment}.enabled}
      enabled: ${features.config-client.${environment}.enabled}
      scheme: https
      host: ${consul_host}
      port: 443
      config:
        fail-fast: true
        acl-token: ${consul_token}
        enabled: ${features.config-client.${environment}.enabled}
        default-context: dev
        prefixes:
          - my/custom/path
        format: FILES
        watch:
          enabled: ${features.config-client.${environment}.enabled}
          delay: 300000
          wait-time: 5