Synchronization does not work.

[tropnikovvl]

Hi!

I am using this configuration for an application.

vaultSync:
  id: vault-sync
  full_sync_interval: 60
  bind: 0.0.0.0:8202
  src:
    url: http://vault-1.vault.svc.cluster.local:8200/
    prefix: ""
    backend: my-backend
    version: 2
    role_id: 7726aexxxx
    secret_id: 48bccb4xxxx
  dst:
    url: http://vault-2.vault.svc.cluster.local:8200/
    prefix: ""
    backend: my-backend
    version: 2
    token: hvs.yxxxx

And in the logs I see that it applies it generally correctly, but for some reason it ignores the backend setting and tries to connect using the default path (path: "/v1/secret/metadata/").

10:17:57 [INFO] Configuration from /config/vault-sync.yaml:                                                                                                                                                                                                                                                                                                           
{                                                                                                                                                                                                                                                                                                                                                                     
  "id": "vault-sync",                                                                                                                                                                                                                                                                                                                                                 
  "full_sync_interval": 60,                                                                                                                                                                                                                                                                                                                                           
  "bind": "0.0.0.0:8202",                                                                                                                                                                                                                                                                                                                                             
  "src": {                                                                                                                                                                                                                                                                                                                                                            
    "url": "http://vault-1.vault.svc.cluster.local:8200/",                                                                                                                                                                                                                                                                                                                                  
    "role_id": "***",                                                                                                                                                                                                                                                                                                                                                 
    "secret_id": "***",                                                                                                                                                                                                                                                                                                                                               
    "token_ttl": null,                                                                                                                                                                                                                                                                                                                                                
    "token_max_ttl": null,                                                                                                                                                                                                                                                                                                                                            
    "prefix": ""                                                                                                                                                                                                                                                                                                                                                      
  },                                                                                                                                                                                                                                                                                                                                                                  
  "dst": {                                                                                                                                                                                                                                                                                                                                                            
    "url": " http://vault-2.vault.svc.cluster.local:8200/",                                                                                                                                                                                                                                                                                                              
    "token": "***",                                                                                                                                                                                                                                                                                                                                                   
    "token_ttl": null,                                                                                                                                                                                                                                                                                                                                                
    "token_max_ttl": null,                                                                                                                                                                                                                                                                                                                                            
    "prefix": ""                                                                                                                                                                                                                                                                                                                                                      
  }                                                                                                                                                                                                                                                                                                                                                                   
}

10:17:57 [WARN] Failed to list secrets in : Error in vault response: Vault request failed: Response { url: Url { scheme: "http", username: "", password: None, host: Some(Domain("vault-1.vault.svc.cluster.local")), port: None, path: "/v1/secret/metadata/", query: None, fragment: None }, status: 403, headers: {"cache-control": "no-store", "content-length": "60", "content-type" 
Could not read vault response.`                                                                                                                                                                                                                                                                                                                                          
10:17:57 [INFO] FullSync finished in 2ms                                                                                                                                                                                                                                                                                                                                 
10:17:57 [INFO] Secrets created/updated: 0, deleted: 0

[pbchekin]

In the debug output you have provided I do not see that backend is set. I have noticed that the helm chart used the old Docker image, I've updated the helm chart to use the most recent version 0.9.2. Could you please try again and also check that vault-sync uses the right configuration file?

[tropnikovvl]

Hi @pbchekin

With the new helm chart everything worked, thank you very much!

Question: is it possible to create backends if they did not exist before synchronization? Perhaps add some kind of flag to this.

My case: I have a main Vault and several in other locations, and after initializing them, I now have to create backends for secrets manually, where I will synchronize the secrets.

[pbchekin]

I understand that this is a valid use case, but looks like it is a Vault administration rather than secrets synchronization. To create a new secrets backend vault-sync needs to have additional privileges and a way to specify a backend parameters, if they differ from the default ones. After that you need to configure permissions for users to use a newly added backend. I would recommend to have a centralized configuration for Vault backends. I believe you have a code that deploys and initializes all that Vault instances, it is a good place to create all these backends and set permissions for the users. This Terraform provider worked for me in the past: https://registry.terraform.io/providers/hashicorp/vault/latest/docs.

[tropnikovvl]

Hi @pbchekin!

I understand. But in my situation, I deploy Vault with Helm, initialize it and transfer the root token to Vault Sync. Thus, the Vault Sync has all the necessary privileges over the remote Vault and can synchronize anything and any way there.

I understand that my situation may not apply to other users.