Spring cloud config server - health check is broken when using vault

[vasilievip]

When I provide vault token, I'm getting keys from vault and git:

curl -X "GET" "http://localhost:8080/myservice.properties" -H "X-Config-Token: 27004d54-f1ac-31eb-572e-7fb1445a6be7"
app.setting1: placeholder1.value
placeholder1: placeholder1.value
vaultkey1: secret

When I go to health check, I'm getting service down status due to missing token when traversing status of vault repo:

http://localhost:8080/health
...
"configServer": {
"status": "DOWN",
"repository": {
"application": "app",
"profiles": "default"
},
"error": "java.lang.IllegalArgumentException: Missing required header: X-Config-Token"
}
....

[spencergibb]

thanks for the report

[vasilievip]

Vault repo requires token from config client. Not sure about design decisions which led to this. Why git integration don't ask config clients for login/password to git repo? Would it be acceptable to add some sort of "deafultVaultToken" and, if config client did not provide one, use default instead?

VaultEnvironmentRepository.java
...
    String read(String key) {

        String token = request.getHeader(TOKEN_HEADER);
        if (!StringUtils.hasLength(token)) {
            throw new IllegalArgumentException("Missing required header: " + TOKEN_HEADER);
        }
...

[ryanjbaxter]

If the git repo requires a username and password you can supply that in the configuration

spring:
  cloud:
    config:
      server:
        git:
          uri: https://github.com/spring-cloud-samples/config-repo
          username: trolley
          password: strongpassword

[spencergibb]

it's specifically the vault token

[vasilievip]

If the git repo requires a username and password you can supply that in the configuration

correct, this user/password will be configured on config server side, but vault integration requires token to be configured on config client side (e.g. each and every from my 100500 services must supply such token :). Maybe it makes sense to setup vault token on config server side for vault as its done for git? This will also fix health endpoint.

[spencergibb]

@vasilievip the point of the vault integration is that each client has a unique token.

[jwcarman]

So, the best practice is to set up a token for each application that needs to use the configuration server? Just trying to understand the intent here. How do folks propose we secure that application-specific token on the client side? Environment variables? Also, when I start up my configuration server, it appears that it is expecting the spring.cloud.vault.token value to be set. It throws this exception:

IllegalArgumentException: Token (spring.cloud.vault.token) must not be empty

Does this mean that I need to supply my Vault token to both the clients and the config server? These tokens should be different?

[ammmze]

So ... the client applications use spring.cloud.config.token to specify what token to use when talking to the config server. Can we just have it where the config server uses that same property when doing the health check? Each client would still need to provide their own token, but this way at least the health check doesn't automatically fail when enabling vault assuming the token you give is a valid token for vault.

FYI ... As a workaround, you can call the health endpoint with the X-Config-Token header and it does pass. In the case of something like kubernetes, i'm not sure if there is a way to pass a variable to the liveness probe.

[ammmze]

Also as referenced in an older issue (#565) it was suggested you can disable the health indicator, which seems kinda wrong...if my app (who's only purpose is to give me configs) can't talk to its datasource that has the configs, that app should be reporting down.

[ryanjbaxter]

This looks like a solution possibly. It appears I can hit this endpoint without the token and get a response back, even if the vault is still locked https://www.vaultproject.io/api-docs/system/health/

[n3y]

I'd welcome having a property holding a Vault token dedicated purely for the purposes of health indicator, i.e. monitoring of Vault done by Spring Cloud Config Server (v2.7.2). For example, some property similar to/inspired by spring.cloud.consul.discovery.health-check-headers.X-Config-Token from spring-cloud-consul would do the job.

Background: I wanted my Spring Cloud Config Server to perform health checks on Git and Vault backends. If I specify spring.cloud.config.server.vault.token in the config-server itself, it checks Vault just fine, but it uses that very token for all requests to Vault, including requests for properties for config clients (regardless of which token they send to config server)... So it seems my only option for now is to turn off the health indicator.