TOKEN auth from file (~/.vault_token)

[tbrantb]

I have many projects which at runtime integrate with vault through KUBERNETES auth like so in my application-prod.yml:

  cloud:
    vault:
      uri: https://vault.prod.acme.com
      namespace: cs
      connection-timeout: 5000
      read-timeout: 15000
      fail-fast: true
      authentication: KUBERNETES
      kubernetes:
        role: myrole
        kubernetes-path: kubernetes-prod/

However, when running locally for development and testing I use TOKEN auth like so in my application-local.yml:

  cloud:
    vault:
      uri: https://vault.dev.acme.com
      namespace: cs
      connection-timeout: 5000
      read-timeout: 15000
      fail-fast: true
      authentication: TOKEN
      token: \${VAULT_TOKEN}

Locally, I obtain a vault token via the vault CLI with OIDC/Okta authentication. All of our developers have this capability to read DEV secrets. I then set the token the vault CLI gives me into the VAULT_TOKEN environment variable. which allows me to gradlew bootrun my application from the same console window just fine. Setting this an an environment variable is somewhat inconvenient if working inside an IDE. I'd need to restart my IDE to have it pick up the latest env variable to use IDE features to run/debug the project.

I thought it would be nice to have a well known file location for the vault token and have spring-cloud-config be able to pull the token for TOKEN authentication from that file. I notice that the Vault CLI places the token value in ~/.vault_token for its use across invocations of commands. I thought rather than exporting that value to an environment variable, if the auth mechanism could just read from this file, then I'd have no need to export the environment variable and worry about which shells/processes need to be restarted to pick up the latest value. Is there a way to customize the TOKEN auth or boot configuration to pull the value from a file rather than static or environment variable?

If it's too confusing, then perhaps another AUTH configuration for TOKEN-FILE?

  cloud:
    vault:
      uri: https://vault.dev.acme.com
      namespace: cs
      connection-timeout: 5000
      read-timeout: 15000
      fail-fast: true
      authentication: TOKEN-FILE
        token-file: file:/etc/loc/tokenfile  (default to ~/.vault_token)

Maybe there's a feature I'm missing that would make this easy already? Thanks for the consideration

[tbrantb]

@mp911de I created a PR here for conversation. I'd love to discuss this feature more to see if its a good idea or other ways to implement. https://github.com/spring-cloud/spring-cloud-vault/pull/611

[mp911de]

Thanks for reaching out. Static tokens can be used directly through spring.cloud.vault.token=… in a properties file or -Dspring.cloud.vault.token=$(cat ~/.vault_token) when using the command line. Introducing another mechanism that does what is already achievable doesn't make much sense.

[tbrantb]

The -Dspring.cloud.vault.token=$(cat ~/.vault_token) doesn't work well for me unless I'm running from a place that can resolve invoke/execute that cat sub-expression. For example executing an integration test from my IDE now has the token missing. I did script the creation of a developer.properties file in my home directory containing spring.cloud.vault.token=…, but I cannot get it to be observed at startup through my profile.

This is the relevant snippet of my application-local.yml. My integration tests et al understand to use the local profile

spring:
  cloud:
    vault:
      uri: https://vault.dev.acme.com
      namespace: cs
      connection-timeout: 5000
      read-timeout: 15000
      fail-fast: true
      authentication: TOKEN
  config:
    import: \${user.home}/developer.properties, vault:///secret/path/db1/dev, vault:///secret/path/db2/dev

When using this local profile, I'd like to load ~/developer.properties which contains the token property:

spring.cloud.vault.token=s.kynZZZZZZZZZZZZZZZ.9f1b2

However, I continue to get java.lang.IllegalArgumentException: Token (spring.cloud.vault.token) must not be empty at startup time.

I thought maybe pulling the spring.config.import above the vault declaration would work, but has no impact.

[tbrantb]

I think I'm struggling a bit with the spring boot 2.4 config changes and spring-cloud-vault and a mixture of confusing documentation... I have the following files: application.yml (base) application-local.yml (for local development) application-dev.yml (for deployment into our dev environment) application-uat.yml (for our uat environment) application-prod.yml (for our prod environment)

Only 1 of local,dev,uat,prod profiles will be activated at a time.

I'm now adding a file in home directory ~/developer.properties that will be populated via a shell script when I login to vault locally. I'd like to have the application load developer.properties only when local profile is activated AND loaded in time for the vault configuration in application-local.yml to see it. Which I THINK means that it cannot be imported by the application-local.yml file itself.

This would seemingly force me to create 2 profiles let's call them local1 and local2, local1 to import the ~/developer.properties and local2 to do the vault configuration among other items like datasources, etc.

As an aside, I noticed that if I create an application-local.properties and do the spring.config.import of developer.properties from there it does work, since springboot loads properties files before yaml files. However, the recommendation is not mix profile specific properties/yaml files.

Am I missing something? Is there a way to import ~/developer.properties which contains the vault token in the same profile specific file that connects to vault and loads properties from vault?

[mp911de]

I wonder whether it would make sense actually to mimic Vault CLI in the sense of checking the presence of the token file if no Vault token is configured. Trying to reconfigure Spring Boot through any other means than provided by the OS/platform is a bit hacky.

[tbrantb]

I can put together a PR for that if you'd like

On Mon, Sep 20, 2021, 4:58 AM Mark Paluch @.***> wrote:

I wonder whether it would make sense actually to mimic Vault CLI in the sense of checking the presence of the token file if no Vault token is configured. Trying to reconfigure Spring Boot through any other means than provided by the OS/platform is a bit hacky.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/spring-cloud/spring-cloud-vault/issues/609#issuecomment-922744251, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACNA7GZ5X3AP7IJV4RWZY3UC3ZTJANCNFSM5EAALKYA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[mp911de]

Sounds good. I think a change in ClientAuthenticationFactory regarding TOKEN authentication should be just fine as ClientAuthenticationFactory is used by both, bootstrap and config data API.

[tbrantb]

Here's the PR: https://github.com/spring-cloud/spring-cloud-vault/pull/614