Add support for YAML files to @PropertySource [SPR-13912]

[spring-projects-issues]

Rob Moore opened SPR-13912 and commented

As the Spring documentation notes, the @PropertySource annotation does not support YAML files so one must use properties files in these cases.

This can be especially annoying with third party libraries using these annotations as a separate properties file must be created.

---

Affects: 4.2.4

Reference URL: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-yaml-shortcomings

18 votes, 27 watchers

[spring-projects-issues]

Stéphane Nicoll commented

Another use case is using i18n with yml (i.e. messages.yml)

[spring-projects-issues]

Keith Hoopes commented

Here's our use case:

1. We are hosting multiple war files on a single tomcat instance (not using boot runner). a. This means we cannot use the spring.config.location and spring.config.name JVM parameters or environment variables.
2. Configuration files are not allowed to be included with the war
3. Any configuration that needs to be modified depending on the environment or server (dev, test, prod for example), must be contained in an external configuration file.
4. Profiles are not allowed (we've had problems with certain admins running apps in prod mode in test, and test mode in prod).
5. Ops/IT must be allowed to specify the location of all configuration files using environment variables that are only accessed by (or are unique to) the particular application to which it belongs.

Example:

// snip: package and import statements
@SpringBootApplication
@PropertySource("file:${myapplication.cofnig.url}"))
public class MyApplication expends SpringBootServletInitializer{
    //snip: content, main, init methods etc...
}

Right now, this approach only works for properties files. It would be really convenient if this would work for either, and let IT decide on the format they best prefer. The annotation should be format agnostic.

[spring-projects-issues]

Gregor Frey commented

I support this request. I used the locations attribute in the ConfigurationProperties annotation, which allowed property files in yaml format. As this attribute is deprecated since SpringBoot 1.4 I'm looking for a simple alternative. To allow yaml files in the PropertySource annotation would be quite helpful.

[spring-projects-issues]

Stéphane Nicoll commented

Gregor Frey there is a reason why it's deprecated, perhaps you could take that into account in your own arrangement? I think there are better options in Spring Boot than this (which means you wouldn't have to wait for this particular feature). An EnvironmnentPostProcessor is a far more powerful option if you don't want to use boot's ways of configuring the environmnent.

But I digress, this is a useful feature in the core indeed.

[spring-projects-issues]

Gregor Frey commented

Hi Stéphane, thanks for pointing me to the EnvironmentPostProcessor. I will definitely have a look.

[spring-projects-issues]

Emerson Farrugia commented

Is there any momentum on this? It seems bizarre that Spring Boot doesn't have a way to compose configuration properties using YAML, which is a first-class configuration mechanism. The justification seems to be that the Spring Framework itself doesn't support it.

My use case is quite simple. I have a library I share amongst microservices to configure logging. I want that same library to contain YAML configuration for logging, e.g. setting logger levels or configuring trace endpoints. Similar examples could be data layer libraries that contain database configurations.

To use YAML, I'm currently limited to using Spring Boot's spring.config.location. Using @PropertySource seems like a saner option.

[spring-projects-issues]

Stéphane Nicoll commented

Ironically enough, that will never work.

The logging system is initialized very early on (before the ApplicationContext is even created). So any @PropertySource you put in your code will be processed way after the logging system has fully initialized. That's the reason there isn't a high incentive to implement this: in a Spring Boot context in particular, @PropertySource is very limited and Spring Boot offers much better alternatives, including EnvironmentPostProcessor that would work with the use case you're describing.

[spring-projects-issues]

Roman Rodov commented

Everyone keeps speaking of these "better alternatives" in Spring Boot :) What exactly are they?

spring.config.location and spring.config.name can be used in the top level application, sure. What about library modules setting these default properties in a yml file and then letting the top level application override them (if needed, or adapt the value from the base module)? How can it be achieved using spring boot without having to do anything in the top level app?

[spring-projects-issues]

Michael Johnson commented

Just a thought. Is there any chance this could be done via a maven plugin that would convert yaml files to properties files at build time?

[spring-projects-issues]

Stéphane Nicoll commented

After much consideration, we're inclined to reject this request.

It feels to us that most voters are using Spring Boot and using @PropertySource is definitely something that we do not recommend. I've just expanded the documentation to include a specific example of loading a YAML file in the environment that should answer the concern.

I am not closing this issue right the way. If you are using Spring Boot and that doesn't answer your concern, please join us on Gitter and we can chat about it. If you're not using Spring Boot, please expand as why you need this at the @PropertySource level.

[spring-projects-issues]

Keith Hoopes commented

I see why @PropertySource would not be proper for loading spring boot auto configuration properties. So really, shouldn't we deprecate the annotation all together and perhaps add a different one that would allow for the property to be loaded before the application context? The example is definitely workable, but it feels somehow out of place in a spring boot application that should "just know" that an "application.yml" file needs to be loaded before the application context.

My problem is that I have to deploy my applications as war files on Tomcat in windows (not my decision), so I have multiple springBoot applications that share the same JVM parameters. This means that I cannot use "-Dspring.config.location=somedir/whatever.properties" nor "-Dspring.config.name=myapp.application.yml" to indicate application properties.

[spring-projects-issues]

Stéphane Nicoll commented

Let's not discuss Spring Boot use case here as I've indicated that we don't consider @PropertySource to be a good fit in light of what Spring Boot has to offer. Deprecating the annotation is not an option (there is really no reason) and replacing it with another annotation is not going to fix anything (same limitation of processing the user configuration once certain things like logging have already completed).

As for your use cases, we can discuss them in the Gitter Spring Boot channel but it boils down to the fact that if you can hardcode the location in your code, you can also hardcode whatever option Spring Boot offers using SpringApplicationBuilder. While @PropertySource may sound interesting because "you add an annotation and you're done", it really doesn't do that so I don't want to increase that impression by offering yaml support for it.

[spring-projects-issues]

Dominik Bartholdi commented

@snicoll do you care to explain why YAML is not going to be supported by @PropertySource? I've read your comments, but I don't understand the rational behind the decision, just because its not to be used in a Boot App does not make the request useless. After all its just a different format and many people like YAML a lot more then the normal Properties format - supporting it does not hurt and its easy to add.

[spring-projects-issues]

Keith Hoopes commented

@snicoll Even though it would not help me do what I was intending, I still agree with @domi. I think perhaps what I was trying to do confused the issue, since I was not understanding the intended use of 'PropertySource'. Since there's no reason to deprecate it, then there's no reason to not enhance it either. Just because some silly developer doesn't understand the best or intended uses, doesn't mean it shouldn't be implemented for those that do. It's the principle of coding with the assumption that everyone knows what they are doing.

I am willing to work on it. Would you accept, or consider accepting, a pull request to add this feature?

[spring-projects-issues]

Stéphane Nicoll commented

Since there's no reason to deprecate it, then there's no reason to not enhance it

I disagree. The whole discussion here is about Spring Boot and how we can "easily" add custom YAML files to the environment. Easy doesn't necessarily means correct. In the context of Spring Boot, the whole use of @PropertySource (be it for properties or potentially yaml based format) is kind of inconsistent and we don't want to promote that feature.

So @PropertySource is a framework feature and for the applications out there using Spring Framework, that's a valid solution. Just because we've rightfully decided to support those users doesn't mean that we have to improve it for new use cases.

Sorry, but we won't accept a PR since we've decided to reject this request.

[spring-projects-issues]

Keith Hoopes commented

Well, thanks for talking with me about it at least. It's been engaging and enjoyable. I'll just leave this last "little" thought and be done.

I was focused only on the Spring Framework here; I had thought we moved away from discussing SpringBoot since you asked to move that conversation to gitter.

In the context of Spring Framework only, it makes sense to include YAML (and it's superset JSON?) in order to provide a complete, quality api. It does not make sense (to me) to allow foreign/outside "wrapper" frameworks to influence the quality and usefulness of the base framework being developed.

In the context of SpringBoot, I don't see why this would cause a problem or send the wrong message; I disagree with the philosophy of preventing developers from doing stuff I don't like (even if I am right).

I won't touch on the discussion of how to specifically load individual SpringBoot configurations for each war in Tomcat, since you and others have provided some information for me I neeed to think about and upon which I need to experiment.

You should probably update the documentation for SpringBoot.

https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html#howto-change-the-location-of-external-properties

I read it several times to be sure, and I had others read it without any prior input, and everyone thought that "If YAML is used then files with the ‘.yml’ extension are also added to the list by default." means that '@PropertySource' already supports YAML. It also states that properties "are added to the Environment early enough to be used in all phases of the ApplicationContext lifecycle", but I am not able to externalize the properties if I need to use all aspects for Spring Boot configuration. Since that is "Spring Boot" documentation, developers would expect it to work for SpringBoot configuration, so it needs some clarification and an update to let us know it is not recommended nor can it be used to load SpringBoot configuration.without any clarifications.

There is documentation at "https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#boot-features-external-config-yaml-shortcomings" that add some clarity, so perhaps a link could be added.

[spring-projects-issues]

Keith Hoopes commented

I would also be willing to work on a PR to update the documentation according to your input/direction if you would like.

https://github.com/Pytry

[spring-projects-issues]

Christian Oestreich commented

I know I am late to this party but our use case is a bit of a case-study in why the support for multiple yml files is important to us.

We make extensive use of Spring Rest Docs and we import a lot of code snippets, test documentation and configuration into the generated site docs. The readability of YML is vastly superior to that of properties files due to its use of spacing to help delineate hierarchy. We have attempted to move all our spring cloud stream bindings to a "bindings.yml" file and include this file into the application via a @PropertySource and in asciidocs via a source include::/bindings.yml. This clearly failed for the reasons above and we had to convert to using a @PropertySource with a bindings.properties file.

Contrary to your above points, we very poignantly want to commonly split configuration up across multiple files so we can use these configuration snippets directly into our documentation and always have accurate and up to date docs. This requires that we externalize and make extensive use of the @PropertySource to pull these in.

I feel that your opinions on the validity of this type of implementation are short sighted and don't take into account creative use scenarios and the fact that YML is a better documentation format and visually easier to read.

Which one would you want to write, maintain, is easier to read and works better with IDEs?

spring:
  cloud:
    stream:
      bindings:
        cedm-consumer:
          destination: xyz-ps-cedm-exchange
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 3
        cedm-producer:
          destination: xyz-ps-cedm-exchange
          group: ps
          contentType: text/plain
        core-input:
          destination: xyz-ps-input-exchange
          group: ps
          contentType: text/plain
          producer:
           requiredGroups: ps
        core-output:
          destination: xyz-ps-output-exchange
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 20
        response:
          destination: xyz-ps-response-status-exchange
          group: ps
          contentType: text/plain
          producer:
            requiredGroups: ps
        error-queue-app:
          destination: xyz-error-queue
          group: ps
          contentType: text/plain
        core-error:
          destination: core-ps-error
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 5
        xyz-input-error:
          destination: xyz-ps-error
          group: ps
          contentType: text/plain
          consumer:
            concurrency: 5
        xyz-output-error:
          destination: xyz-ps-error
          group: ps
          contentType: text/plain

spring.cloud.stream.bindings.cedm-consumer.destination=xyz-ps-cedm-exchange
spring.cloud.stream.bindings.cedm-consumer.group=ps
spring.cloud.stream.bindings.cedm-consumer.contentType=text/plain
spring.cloud.stream.bindings.cedm-consumer.consumer.concurrency=3
spring.cloud.stream.bindings.cedm-producer.destination=xyz-ps-cedm-exchange
spring.cloud.stream.bindings.cedm-producer.group=ps
spring.cloud.stream.bindings.cedm-producer.contentType=text/plain
spring.cloud.stream.bindings.core-input.destination=xyz-ps-input-exchange
spring.cloud.stream.bindings.core-input.group=ps
spring.cloud.stream.bindings.core-input.contentType=text/plain
spring.cloud.stream.bindings.core-input.producer.requiredGroups=ps
spring.cloud.stream.bindings.core-output.destination=xyz-ps-output-exchange
spring.cloud.stream.bindings.core-output.group=ps
spring.cloud.stream.bindings.core-output.contentType=text/plain
spring.cloud.stream.bindings.core-output.consumer.concurrency=20
spring.cloud.stream.bindings.response.destination=xyz-ps-response-status-exchange
spring.cloud.stream.bindings.response.group=ps
spring.cloud.stream.bindings.response.contentType=text/plain
spring.cloud.stream.bindings.response.producer.requiredGroups=ps
spring.cloud.stream.bindings.error-queue-app.destination=xyz-error-queue
spring.cloud.stream.bindings.error-queue-app.group=ps
spring.cloud.stream.bindings.error-queue-app.contentType=text/plain
spring.cloud.stream.bindings.core-error.destination=core-ps-error
spring.cloud.stream.bindings.core-error.group=ps
spring.cloud.stream.bindings.core-error.contentType=text/plain
spring.cloud.stream.bindings.core-error.consumer.concurrency=5
spring.cloud.stream.bindings.xyz-input-error.destination=xyz-ps-error
spring.cloud.stream.bindings.xyz-input-error.group=ps
spring.cloud.stream.bindings.xyz-input-error.contentType=text/plain
spring.cloud.stream.bindings.xyz-input-error.consumer.concurrency=5
spring.cloud.stream.bindings.xyz-output-error.destination=xyz-ps-error
spring.cloud.stream.bindings.xyz-output-error.group=ps
spring.cloud.stream.bindings.xyz-output-error.contentType=text/plain

Every developer I have asked on our team prefers to look at and use the YML, especially in documentation. Have you opened this discussion up to the broader audience and allowed community input via a poll or other mechanism concerning supporting YML files and @PropertySource?

So @PropertySource is a framework feature and for the applications out there using Spring Framework, that's a valid solution. Just because we've rightfully decided to support those users doesn't mean that we have to improve it for new use cases.

This comment just makes no sense and is like saying: "Well... we invented a wheel, but you can only use it for wheelbarrows as we don't feel that it's usefulness extends beyond that."

[spring-projects-issues]

Claudiu Alexandru Popa commented

Sorry. I know it's a little bit to late... Or, on the contrary, it's still very valid...

But, we are now (at the time I am writing this) at Spring version 5.0.7, and @PropertySource still exists in spring.

All the discussion is not, in my opinion, about spring-boot. It is about Spring Framework. 

And since we have @PropertySource in Spring Framework, which can be used with a .properties file, I see no reason to not be used with an .yaml (.yml) file.

That's why, until the @PropertySource is not completely removed, I consider that the feature of using it with properties file or yaml file (at user disposal's) should be offered.

I have already implemented (just for myself) this.

Please, can you evaluate the possibility to integrate my changes into the main branch?

[spring-projects-issues]

Bulk closing outdated, unresolved issues. Please, reopen if still relevant.

[mistriel]

I strongly agree with the supporters of this feature !
YAML files should be supported as 1st class citizens in Spring F.W.

Should be reopened.

[maxmin13]

crap

[mazine-rr]

.yaml is far more enjoyable to write, read and modify than .properties. Please reopen.

[raahool]

My Use case is,

1. organisation wide all our microservies are using YAML, so we cannot move from YAML to .properties.
2. In one of my recent app, I am using testcontainers for Unit tests, where testcontainers randomly selects any available port to start the container. So before running the tests I have to dynamically change the PORT property which is defined in the YAML file.

For .properties file, it was straightforward with ApplicationContextInitializer& TestPropertyValues- like TestPropertyValues.of("spring.rabbitmq.port" + container.getMappedPort(5672)).applyTo(applicationContext);

Not able to achieve this, being the port property is defined in the YAML file.

Any workaround / suggestions ?

[dlehammer]

I've included a working example (Spring Boot 2.7.0) for posterity below. To support those like me puzzled by the below quote from the documentation, as there's no elaboration or example.

You can also use the YamlPropertySourceLoader class if you want to load YAML as a Spring PropertySource.

(I haven't pursued the YamlPropertySourceLoader approach, as I sense the time to deduce the "right approach" and the boiler-plate far exceeds the example I've provided) The use-case in my case is simple, load a external .yml file into a @ConfigurationProperties annotated class ~ in this case the configuration contains some lists, which are readable in YAML less so in properties.

Seemingly in spite of "Easy doesn't necessarily means correct" this a desired approach, that has spawned tutorials as recent as 2022-07-22 (@PropertySource with YAML Files in Spring Boot).

With a PropertySourceFactory like below (ex. from the Baeldung tutorial), the object binding is performed similarly to properties for a @PropertySource(value = "classpath:foo.yml", factory = YamlPropertySourceFactory.class)

public class YamlPropertySourceFactory implements PropertySourceFactory {

    @Override
    public PropertySource<?> createPropertySource(String name, EncodedResource encodedResource) throws IOException {
        YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean();
        factory.setResources(encodedResource.getResource());

        return new PropertiesPropertySource(encodedResource.getResource().getFilename(),  factory.getObject());
    }
}

I too was surprised to discover (via this obscure issue) that Spring YAML support is missing feature-parity with properties ~ ie. consistently supported via externalized configuration, this violates the principle of least astonishment and subsequently spawns unofficial workarounds :/

I vote for an official approach 👍

See also similar tutorials

• Use @PropertySource with YAML files (2018-07-04)
• Spring @PropertySource using YAML (2021?)

[gavenkoa]

There are two SO discussion dedicated to the report:

• https://stackoverflow.com/questions/45880494/configurationproperties-loading-list-from-yml/
• https://stackoverflow.com/questions/21271468/spring-propertysource-using-yaml/

@PropertySource(value = "classpath:XXX.yml", factory = ...) is a nice solution though you need to create own PropertySourceFactory (as @dlehammer presented). The best thing, in light of Spring dev denial, is to prepackage YamlPropertySourceFactory implementation so no one needs to copy-paste some non-bullet proof implementation but relies on good default!

@TestPropertySource doesn't support a factory but has a property to be loaded with highest priority ((

[hbourada]

What about a new annotation @YamlPropertySource with meta annotation @PropertySource on it with the correct factory as bellow:

@PropertySource
public interface @YamlPropertySource {
        @AliasFor(annotation = PropertySource.class)
        String name() default "";

        @AliasFor(annotation = PropertySource.class)
    String[] value();

        @AliasFor(annotation = PropertySource.class)
    boolean ignoreResourceNotFound() default false;

        @AliasFor(annotation = PropertySource.class)
    String encoding() default "";

        @AliasFor(annotation = PropertySource.class)
    Class<? extends PropertySourceFactory> factory() default YamlPropertySourceFactory.class;
}