Allow application properties to be contributed from module jars

[philwebb]

Currently application.properties and application.yaml files are loaded from the classpath and the first match wins. This makes it hard to bundle common properties into jar files. It would be nice if we could find a way for jars to also contribute properties without needing to implement a EnvironmentPostProcessor.

There are a few things we need to be careful of:

• What order to we load the resources and how do we make sure a developer can override a contributed value
• We don't want to increase startup time (for example by doing a full classpath search)
• We can't break back-compatibility
• We need to be mindful of spring.config.import=classpath:

[mbhave]

Ordering will be tricky with this one. If two jars happen to provide application.properties files with the same value, we need a way to determine which one will win.

[alex-lx]

Would using an order property in application.properties be a simple way?

[philwebb]

We should look into the use-case behind #25033 when we fix this one.

[stategen]

@philwebb & @wilkinsona ,thanks for your great work!

--spring.config.intergration-location=classpath*:/sink//[application.yml], in my pull request #25082, similar #25080

the dependency jar's configs just to reduce the duplicate or mistake works by the the project who use it. so the rule: lowest order and can be overrided by any, not care conflicts(use map value),not overrides higher/outside value, limit 2 wildcards ,endswith '/' or filename and filtered by '/fold/' . the rule will never break back-compatibility. and with a new arg to ensure never break back-compatibility. In my real project（240 jars） with the update i pulled, the startup time is not slow,just normally. I perfer to use spring in xml(`<resource import="classpth:">`), but more and more cloud-framework only provide with spring-boot'style config,there is really no back-way to use spring-boot

[stategen]

I perfer to use spring in xml() = I perfer to use spring in xml(< resource import="classpath:*">)

[wilkinsona]

We should also consider https://github.com/spring-projects/spring-boot/issues/25084 when looking at this.

[jeffbswope]

I was mulling something along the lines of #25084 like making spring.config.name multi-value (or an additional property) and letting libraries contribute their own values to that.

Maybe all becomes absurd at some point and ordering/precedence remains difficult but for most of our cases anyway you wouldn't expect collisions/conflicts since the libraries own their own vertical pieces.

Perhaps end of the day what you need is not much easier than putting in an EnvironmentPostProcessor for the library-developer, but ends up more powerful and gives a neater solution for some of the rough edges we have in our various stacks ensuring apps get all/only the properties they need and various secrets/settings are in a single location.

[stategen]

The characteristics of spring-boot-cloud-based project are as follows:

1. Distributed, loose and uncertain integration,
2. The provider and consumer are clients of each other, and there are many communication and data protocol configurations.
3. The relationship between the provider and the consumer is many to many. As a result, the exposed configuration is different, which determines that it is not advisable to obtain the configuration by filtering name, and there is the problem of backward compatibility. and hard to learn.
4. Development is distributed in different teams, one of which is only specialized in its own configuration. provide interfaces and configuration they need.
5. Data transmission and consumption are based on interface.
6. Therefore, with the increase of iterative development times, it is more and more difficult to copy the configuration from the dependency party. This kind of work is negligent, slack, and the testing work is huge.

Many distributed development teams are obsessed with configurations they are not familiar with. In fact, they don't need to care about these configurations. They only need default values and full control.

I've been thinking about this for a long time. It's a matter of architecture and foundation. In some interviews, their team needs a lot of people to develop, maintain and synchronize their configuration, which is not needed.

I read the spring boot source code, do a lot of experiments, at last find the best solution I think, my changes #25082, #25080 impact point is the smallest. Fully compatible with your concerns.

[mauromol]

My main use case is splitting configuration properties into multiple files to avoid having a giant application.properties file. Not necessarily these multiple properties files may come from module jars, but that's a possibility of course. Indeed, different properties files may be used just to group configuration properties in a logical way.

In Sprig Boot 2.3 I somewhat overcome this problem with something like this:

@PropertySource(
        ignoreResourceNotFound = true,
        value = { "classpath:config/authentication.properties",
                "classpath:config/authentication-${spring.profiles.active}.properties",
                "classpath:config/persistence.properties",
                "classpath:config/persistence-${spring.profiles.active}.properties" })

that is, since PropertySources are not profile aware (see #12822, which was rejected), I had to mimic the mechanism, setting ignoreResourceNotFound to true.

Now with Spring Boot 2.4 I can probably do something like this directly with spring.config.import (I've not tried yet), but once again I have to somewhat "mimic" the profile specific behaviour Spring Boot reserves to just application.properties, as well as the other mechanism by which an application.properties file outside a bundled JAR will take precedence over one inside the JAR.

What about something like this:

spring.config.units=authentication,persistence

which does something like this:

• search for authentication[-*].properties with the same algorithm you use for application.properties (including support for profiles and paths outside the JAR)
• search for persistence[-*].properties with the same algorithm you use for application.properties (including support for profiles and paths outside the JAR)
• with regards to order, make application.properties[-*].properties win over the imported units (this should cover the "import default properties from a module JAR" case)
• among the imported units, preserve the order in which they are declared (i.e.: make authentication[-*].properties win over persistence[-*].properties)

Just my 2 cents.

[asarkar]

How about adding an attribute to the @ConfigurationProperties, named defaults, that'll take a list of properties not unlike the properties attribute in @SpringBootTest does? The default properties will not need to be prefixed, since the @ConfigurationProperties already does that. This way, libraries don't need to create an additional property file.

[wilkinsona]

We already have support for configuring a property's defaults by initialising the field's value or using @DefaultValue. In both cases those defaults will be reflected in the configuration property metadata and shown during auto-completion in your IDE.

We had a location attribute on @ConfigurationProperties in 1.x, that allowed the location of an external file to be specified. Experience showed that it didn't work particularly well and caused quite a bit of confusion (for example https://github.com/spring-projects/spring-boot/issues/5111 and https://github.com/spring-projects/spring-boot/issues/5135) so it was deprecated and then removed. An attribute that allows properties to be set directly would reintroduce the same problems which we don't want to do. It also wouldn't be as broadly useful as easily adding whole properties or yaml files to the environment.

[asarkar]

a property's defaults by initialising the field's value or using @DefaultValue

I wasn't aware of the @DefaultValue annotation, it seems to have been introduced in 2.2.0. Does it, or inline initialization, allow for placeholders using SPEL?

[wilkinsona]

No, it doesn't. Type conversion is performed – so, for example, you can have a default value of 10s on a Duration property – but there's no placeholder resolution or SpEL expression evaluation.

If you have any further questions, please follow up on Stack Overflow or Gitter. As mentioned in the guidelines for contributing, we prefer to use GitHub issues only for bugs and enhancements.

[philwebb]

See #27000 for another example of someone wanting to contribute config.

[wilkinsona]

https://github.com/spring-projects/spring-boot/issues/27544 is another one to consider when looking at this.

[victorherraiz-santander]

could it be possible to allow merging default properties instead of replacing them?

https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/SpringApplication.html#setDefaultProperties-java.util.Map-

Adding a new method to allow add instead of replace?

After that, something like defining some known property files in the jars could contribute to configure defaults.

At the moment I am doing something like:

public class MyEnvironmentPostProcessor implements EnvironmentPostProcessor {

    @Override
    public void postProcessEnvironment(ConfigurableEnvironment environment,
                                       SpringApplication application) {
        try {
            environment.getPropertySources().addLast(
                new ResourcePropertySource("management-defaults",
                    "/somepath/management.properties"));
        } catch (IOException e) {
            throw new UncheckedIOException(e);
        }
    }
}

[NicoStrecker]

can i somehow weaken the priority of a profile @wilkinsona ?

I want to include properties of my application-customprofile.yml but only if they are not defined in my application.yml

Edit: That seems to work

Application

library:
   customize:
      url:
         firstUrl: "https://app-overriding-url.com"

Library:

library:
   url:
      firstUrl: '${library.customize.url.firstUrl:https://librarys-own-url.com}'

or Library fancy:

library:
   defaults:
      url:
         firstUrl: "https://librarys-own-url.com"
   url:
      firstUrl: '${library.customize.url.firstUrl:${library.defaults.url.firstUrl}}

might help someone 😄