Support .env file for configuration properties

[linux-china]

Lots of developers use .env to process own configuration for script or app. Now I add spring.config.import in application.properties as following:

spring.config.import=optional:file:.env

and SpringBoot app throws the following exception:

Caused by: java.lang.IllegalStateException: File extension is not known to any PropertySourceLoader. If the location is meant to reference a directory, it must end in '/'
    at org.springframework.boot.context.config.StandardConfigDataLocationResolver.getReferencesForFile(StandardConfigDataLocationResolver.java:199)
    at org.springframework.boot.context.config.StandardConfigDataLocationResolver.getReferences(StandardConfigDataLocationResolver.java:121)

I think .env file could be considered as properties file for spring.config.import .

[scottfrederick]

@linux-china Thanks for getting in touch. Simply considering a .env file as a properties file wouldn't do what I think most users expect.

When Spring Boot reads key/value pairs from the system environment, additional processing is required to map the typical UPPERCASE_WITH_UNDERSCORES style of environment variable keys to the canonical form of Spring Boot property keys. This mapping is not done when configuration is read directly from a properties file. I suspect most users would use the environment variable style of keys in a .env file, so the fact that the key name mapping is not performed when importing the file as a properties file would result in the expected properties not being present in the Spring Environment.

In order for this to work as expected, a .env file would need to be recognized as a file containing key/value pairs similar to system environment variables, and processed accordingly. This is, of course, a bigger change than what you've suggested.

Can you elaborate on the types of properties that you would find convenient to set in a .env file instead of a local application.properties file?

[linux-china]

@scottfrederick Now I use jbang to write some scripts for Spring Boot App, scripts and app share some same configuration, such as database url etc, and https://github.com/cdimascio/dotenv-java is very friendly for jbang and kscript. I also have some script managed by just command runner, and .env is also supported by oh-my-zsh dotenv plugin, and .env variables will be loaded into env variables automatically for some command line tools. I mean .env is convenient for developers.

But now still ok for me, I can use ln -s .env env.properties and spring.config.import=file:env.properties also works well. Or ln -s .env application.properties for a local application.properties for Spring Boot.

[scottfrederick]

@linux-china Here's a more concrete example of what I think would be the problem with loading a .env file as a properties file:

In a properties file, you can set a property like spring.application.name=myapp. If you want to set this property as an environment variable, you can set SPRING_APPLICATION_NAME=myapp. Either property will resolve to spring.application.name in the Spring Environment (e.g. using @Value(${spring.application.name})).

If you have SPRING_APPLICATION_NAME=myapp in a properties file, this will not be resolved to spring.application.name because the property key mapping logic is not applied when reading properties files. Properties file are expected to use the canonical form.

Since .env files typically contain keys that conform to the environment variable naming conventions, I think users would naturally set properties like SPRING_APPLICATION_NAME=myapp in the .env file and then be surprised when the application name wasn't set as expected (because the file is being read as a properties file and the property key mapping isn't done). This will work if you set spring.application.name=myapp in the .env file, but you'd have to understand how Spring Boot works to know to use that form.

I can use ln -s .env env.properties and spring.config.import=file:env.properties

In your use case, are you using the canonical form in .env, or how are you setting the properties there?

[linux-china]

yes, I use normal key style form in .env file, and my .env file content as following:

nick=linux_china
management.server.port=9999

then I create link for .env with ln -s .env env.properties

my application.properties as following:

spring.application.name=spring-boot24-demo
spring.config.import=optional:file:env.properties

All work well, @Value("${nick}") private String nick; and management listen port is 9999

but one problem is the env variable, management.server.port could not be used as env variable name, and you have described this problem. Maybe I should do some modification for dotenv plugin, and convert management.server.port to MANAGEMENT_SERVER_PORT automatically.

[scottfrederick]

We think there is value in supporting .env files as an enhancement to Spring Boot. We've marked the issue pending-design-work to consider a few implementation options:

• Automatically process files named .env in the current directory
• Add a new ConfigDataLocationResolver and ConfigDataLoader to support loading files with something like spring.config.import=envfile:.env (note that this would read the contents of the file as a set of environment variables, not as a properties file as originally suggested)
• Add support for a new extensionless-file hint type for files containing environment variable key-value pairs with something like spring.config.import=file:/etc/config/.env[.env]

[linux-china]

@scottfrederick I mad a mistake about .env file format, and key should be uppercase like following:

KEY=xxx
DATABASE_URL=xxxxx

Some frameworks use multi .env files like Spring Boot profile, for example Vue.js CLI.

.env               
.env.local         
.env.[mode]   

I'm not sure some developers use following style or not.

.env
local.env
xxxx.env

Maybe Spring Boot should support to load .env.dev or dev.env styles by profile name.

[RogerVFbr]

I think maybe the better approach would be to parse the .env file upon application startup and provide the content as "virtual" environment variables, whose keys could be addressed in application.properties by regular placeholders. As such:

.env file content: DB_USER=mockUserName

application.properties content: spring.datasource.hikari.username=${DB_USER}

[TheBestPessimist]

If anyone's still interested in using .env as another properties file, for me it works by adding

# Development properties (also used for docker-compose)
spring.config.import=optional:file:.env[.properties]

into application.properties.

The [.properties] is a hint for spring which translates to "Use this file without an extension as a .properties file."

[linux-china]

If anyone's still interested in using .env as another properties file, for me it works by adding

# Development properties (also used for docker-compose)
spring.config.import=optional:file:.env[.properties]

into application.properties.

The [.properties] is a hint for spring which translates to "Use this file without an extension as a .properties file."

It works, but still some work for Spring Boot. Spring Boot should consider additional processing is required to map the typical UPPERCASE_WITH_UNDERSCORES style of environment variable keys in .env file.

[x80486]

One consideration that always comes up is that environment variables will impact all profiles, and while this sort of makes sense, it's sometimes tricky when running tests and things like that.

I think the implementation should take into consideration a way to set some environment variables for a specific profile — something like:

#
#
the_boot.ENV_VAR = value # ...will only impact the_boot profile

ENV_VAR_ALL = world      # ...will apply at application level (like a usual OS environment variable declared and accessible anywhere)

[spam-n-eggs]

Not trying to disrupt anyone else's work here, but there's a package for that: https://search.maven.org/artifact/me.paulschwarz/spring-dotenv/2.3.0/jar

[maranza]

I think maybe the better approach would be to parse the .env file upon application startup and provide the content as "virtual" environment variables, whose keys could be addressed in application.properties by regular placeholders. As such:

.env file content: DB_USER=mockUserName

application.properties content: spring.datasource.hikari.username=${DB_USER}

How do i "parse the .env file upon application startup and provide the content as "virtual" environment variables" So that i can read the values inside .env and use them on my application.properties file, i want to store my password on the .env than directly on the application.properties file. Please give me pointers. thanks

[x80486]

The co.uzzu.dotenv.gradle Gradle plugin does that (in that context). If you need to read the environment variable values in the source code, you would need to use a library: io.github.cdimascio:dotenv-java.

I hope that Spring Boot, at some point, unifies all of that "natively".

[maranza]

Thank you so much. However maven seems to not be able to find the co.uzzu.dotenv.gradle from the repos. I am getting the following error

co.uzzu.dotenv.gradle:co.uzzu.dotenv.gradle.gradle.plugin:pom:1.1.0 was not found in https://repo.maven.apache.org/maven2

when trying both:

```

    <dependency>
        <groupId>co.uzzu.dotenv.gradle</groupId>
        <artifactId>co.uzzu.dotenv.gradle.gradle.plugin</artifactId>
        <version>1.1.0</version>
        <type>pom</type>
    </dependency>

and 

    <dependency>
        <groupId>co.uzzu.dotenv</groupId>
        <artifactId>gradle</artifactId>
        <version>1.0.1</version>
    </dependency>

Which i got from:

`https://mvnrepository.com/artifact/co.uzzu.dotenv/gradle/1.1.0`

[wilkinsona]

@maranza Even with the right repository configuration, you won't be able to use a Gradle plugin in Maven.

[maranza]

Hi @wilkinsona thank you for clarity. so what i am trying to achive is currently not possible. loading application.properties value from a .env file?

[wilkinsona]

It's possible, but you'd have to write some code of your own to parse the .env file and load it as configuration data. The io.github.cdimascio:dotenv-java library that @x80486 linked to above may help you to do that.

[membersound]

If anyone's still interested in using .env as another properties file, for me it works by adding

# Development properties (also used for docker-compose)
spring.config.import=optional:file:.env[.properties]

into application.properties.

The [.properties] is a hint for spring which translates to "Use this file without an extension as a .properties file."

It would be great if this could work with spring.profiles.active to fetch the correct .env, like: spring.config.import=optional:classpath:.env-${spring.profiles.active}[.properties]

[TheBestPessimist]

@membersound since spring.profiles.active has a list of values, which .env should spring load? imo this option is extremely difficult to implement for arguably little benefit

[membersound]

I forgot that multiple profiles can be added to that property (as I always only use one profile at once).

But well, in general the setup might be as follows:

application.properties
application-test.properties
application-production.properties

Now if each of the environments contains secrets that could be externalized into a .env file, it would result in something like:

.env.example    #listing all values that should be configured by the user. should not be loaded by spring, and is commited.
.env            #general secrets that are valid for all profiles
.env-test       #profile specific values
.env-production #profile specific values

How would you tell spring to load first the .env, and then the profile-specific env?

Of course every application-*.properties could set the spring.config.import property itself, but I'd prefer having a one-line statement in only one file that covers all situations. (because normally we have some kind of application-commons.properties that all of our projects inherit from. And this commons project should define the spring.config.import value for all .env files, so that implementing projects don't have to care about .env configuration, but can simply throw them into the /src/main/resources folder and it works).

[x80486]

A .env file is usually meant to be used by you only when working in your desktop/laptop. On higher environments you just refer to the environment variables and act accordingly.

[membersound]

If so, where do you put secrets for test/production then (assuming a locally build app)? At least they cannot be written into the application-*.properties, as those are usually committed.

[x80486]

No, you refer to those secret values by using an environment variable in your application-*.properties files:

# .env file
DATABASE_PASSWORD = "ThePassword"

SPRING_PROFILES_ACTIVE = "production"

# application.yaml
spring:
  datasource:
    password: "${DATABASE_PASSWORD}"

How do you fetch and provide those secrets is outside the scope of this issue. You can as well create a .env file and source it with those values before starting the application — while using the same variables names.

If you have multiple .env[-profile] files you will have the same problem as having those secrets in the usual .properties files: where are you going to store them securely? On the other hand, if you have the means to store and retrieve secrets securely for each environment, but you can't provide them as environment variables, you could inject them by following the using the usual .env.template file and replacing the values accordingly — resulting in just a .env file in the end.

[reflexdemon]

I would like to see the .env file be used to setup development environment. These custom variables will be referred in my local development profile.

My typical usecase for .env would be to have it on the project home directory.

File: .env

MY_KEY=My Local Value
...

On my local application.properties or application.yml file, would use it like,

my.app.properties=${MY_KEY} is working
. . .

Or

my:
 app:
    properties: '${MY_KEY} is working'

And expect my code using @Value("${my.app.properties}") to be able to resolve it before doing DI.

[membersound]

I would like to see the .env file be used to setup development environment. These custom variables will be referred in my local development profile.

That would indeed be the ideal solution.

[petromir]

Any plans to implement suggested by @reflexdemon solution soon?

[philwebb]

@petromir not really I’m afraid. We’re currently investing most of time on other issues. I suspect it might take a while for us to get to this one.

[JayDi85]

How to use env files with application.yml config:

spring:
  config:
    import: optional:file:app.env[.properties]

  datasource:
    url: ${DATABASE_URL}
    username: ${DATABASE_USER}
    password: ${DATABASE_PASSWORD}

[bdmstyle]

How to use env files with application.yml config:

spring:
  config:
    import: optional:file:app.env[.properties]

  datasource:
    url: ${DATABASE_URL}
    username: ${DATABASE_USER}
    password: ${DATABASE_PASSWORD}

works fine!

[membersound]

I can confirm the spring.config.import: optional: approach works fine both for local development and automated deployments, as follows:

With application.properties:

spring.config.import=optional:classpath:.env[.properties]
spring.datasource.password=${DATABASE_PASSWORD}

Now you can create an .env file only on your local development machine with:

DATABASE_PASSWORD=secret

As well, you can use your CI/CD deployment jobs to provide the secret from secret-store, and pass it to the spring application as environment variable, eg with docker-compose:

services:
  app:
    environment:
      - DATABASE_PASSWORD=$DATABASE_PASSWORD

Unfortunately, it was not possible o add the spring.config.import=optional:classpath:.env[.properties] definition to a company-commons.properties file in a company-commons.jar library that is used by the implementation applications. The spring.config.import setting must be repeated for any application explicit, and somehow cannot be inherited.

[petromir]

I can confirm the above approach works as well! Just be careful when you have double quotes in your string, e.g.

SOME_JSON_VALUE_WITH_DOUBLE_QUOTES={"key":"\\"value with empty quotes\\""}

Double quotes should be simply escaped with \\

@philwebb @reflexdemon Can we document

spring.config.import=optional:classpath:.env[.properties]
spring.datasource.password=${DATABASE_PASSWORD}

somewhre, e.g. https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.external-config.files.optional-prefix or https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#features.external-config.files.property-placeholders. I would be happy to do a PR for this :)

[philwebb]

I'm glad that the approach works, but I'm a little wary of recommending it in the docs. It's working by accident rather than design and although .properties and .env files look similar, I think there are some subtle differences (for example, the double quotes issue). I'd rather leave the docs as they are until we implement proper support.

[maciejwalkowiak]

Till there's a first class support in Spring Boot for .env files, https://github.com/paulschwarz/spring-dotenv does it well.

[adSesugh]

How to use env files with application.properties config:

1. Add this dependency
**For Maven**

<dependency>
    <groupId>me.paulschwarz</groupId>
    <artifactId>spring-dotenv</artifactId>
    <version>2.3.0</version>
</dependency>

**For Gradle**

dependencies {
    implementation 'me.paulschwarz:spring-dotenv:4.0.0'
}

2. Config you application.properties like the below

spring.datasource.url={DATABASE_URL}
spring.datasource.username=${DATABASE_USER}
spring.datasource.password=${DATABASE_PASSWORD}

3. Create .env file in your resources directory with the appropriate content;
DATABASE_URL=
DATABASE_USER=
DATABASE_PASSWORD=

Enjoy :)

[petromir]

@maciejwalkowiak @adSesugh we know about this project, but the point is to have it as part of Spring Boot, as for me it looks like a basic functionality.

[jonatan-ivanov]

My two cents: To me it does not seem like a good idea in general for the application to do this. In the first place, I would try to avoid having a .env file at all and use profiles to have my dev/local/default properties set, e.g.: application-local.properties locally and application-prod.properties in prod. Or application.properties locally and application-prod.properties in prod. If you need to load secrets locally, you may still be able to use global devtools properties. If this does not work for your properties (different apps has different needs), I would use a tool to manage the project-related environment variables for me, like https://direnv.net. What it does is loading env vars from the file when you cd into a directory, it also makes possible that these environment variables are available for other processes too like your build tool, docker, etc).

[renannprado]

My two cents: To me it does not seem like a good idea in general for the application to do this. In the first place, I would try to avoid having a .env file at all and use profiles to have my dev/local/default properties set, e.g.: application-local.properties locally and application-prod.properties in prod. Or application.properties locally and application-prod.properties in prod. If you need to load secrets locally, you may still be able to use global devtools properties. If this does not work for your properties (different apps has different needs), I would use a tool to manage the project-related environment variables for me, like https://direnv.net. What it does is loading env vars from the file when you cd into a directory, it also makes possible that these environment variables are available for other processes too like your build tool, docker, etc).

I don't use .env profile in production (or any other environment actually), but I do use it for local development.

Just as one of the examples I have, I have a .env file which contains password for the database (postgres) which is started via docker compose. By using this .env (local.env) file, I'm able to set the password for the database in docker compose AND use the exact same property in my application-local.yaml via ${DATABASE_PASSWORD} template var in my local spring profile.

So yes, IMHO it is a good idea, at least for development purposes I can see that. For usage inside of kubernetes/production, then I cannot see why one would need it.

[x80486]

My two cents: To me it does not seem like a good idea in general for the application to do this. In the first place, I would try to avoid having a .env file at all ...

That's true for almost every profile, except the one you are going to use to test/run the application locally. Probably having an opt-in configuration set makes more sense, and developers will need to configure it and enabled it when desired — preferably only for that "local" profile.

devtools will only work if you have a single application — or all applications share the same set of key/value pairs. The direnv may fail in sandboxed environments — whereas a simple .env file checks all the boxes, and it's really becoming the standard these days on many platforms/runtimes.

Will be even better if Spring Boot will come with a predefined set of profiles (dev, test, prod), where for instance, dev activates certain enhanced functionalities by default — usually towards local development and such.

[rodneyazev]

My two cents: To me it does not seem like a good idea in general for the application to do this. In the first place, I would try to avoid having a .env file at all ...

That's true for almost every profile, except the one you are going to use to test/run the application locally. Probably having an opt-in configuration set makes more sense, and developers will need to configure it and enabled it when desired — preferably only for that "local" profile.

devtools will only work if you have a single application — or all applications share the same set of key/value pairs. The direnv may fail in sandboxed environments — whereas a simple .env file checks all the boxes, and it's really becoming the standard these days on many platforms/runtimes.

Will be even better if Spring Boot will come with a predefined set of profiles (dev, test, prod), where for instance, dev activates certain enhanced functionalities by default — usually towards local development and such.

I think the issue here is that we have the possibility of an application.properties file (default, dev, test or prod) being able to recognize a .env environment variable file that can be used by both the Spring Framework (via application.properties ) and Docker (via Dockerfile or Docker-Compose). This would make our lives a lot easier.

P.S.: I'm using https://mvnrepository.com/artifact/me.paulschwarz/spring-dotenv library. It works like a charm.

[hendisantika]

When run mvn clean package it will look for JDBC configs.

How to fix this using

<dependency>
    <groupId>me.paulschwarz</groupId>
    <artifactId>spring-dotenv</artifactId>
    <version>2.3.0</version>
</dependency>

@adSesugh

Thanks

[ashwin31]

My two cents: To me it does not seem like a good idea in general for the application to do this. In the first place, I would try to avoid having a .env file at all ...

That's true for almost every profile, except the one you are going to use to test/run the application locally. Probably having an opt-in configuration set makes more sense, and developers will need to configure it and enabled it when desired — preferably only for that "local" profile. devtools will only work if you have a single application — or all applications share the same set of key/value pairs. The direnv may fail in sandboxed environments — whereas a simple .env file checks all the boxes, and it's really becoming the standard these days on many platforms/runtimes. Will be even better if Spring Boot will come with a predefined set of profiles (dev, test, prod), where for instance, dev activates certain enhanced functionalities by default — usually towards local development and such.

I think the issue here is that we have the possibility of an application.properties file (default, dev, test or prod) being able to recognize a .env environment variable file that can be used by both the Spring Framework (via application.properties ) and Docker (via Dockerfile or Docker-Compose). This would make our lives a lot easier.

P.S.: I'm using https://mvnrepository.com/artifact/me.paulschwarz/spring-dotenv library. It works like a charm.

it is a great one until you deploy. i tried it to load env variables and it worked excellent in development. env variables are not being picked up once i build my spring boot project without supplying env variables at the build time. i need the jar file to run in different environments with different configurations. I don't like to supply .properties file for obvious reasons (to deploy in kubernetes, etc). can any one give me info one how to load env variables after building jar file.

[wilkinsona]

can any one give me info one how to load env variables after building jar file

Typically, you would configure the variables in the operating system's environment. In Kubernetes, one way to do this is described here. If you have any further questions, please follow up on Stack Overflow as this is largely off-topic for this issue.