search

redhat-cop / vault-config-operator

An operator to support Haschicorp Vault configuration workflows from within Kubernetes
Apache License 2.0
108 stars 49 forks source link

VaultSecret - ```missing role``` #226

Closed jimsmith closed 7 months ago

jimsmith commented 7 months ago

Hello,

Vault Config Operator: 0.8.25

We are spiking this and we have the need to be able to point VaultSecret to a Vault Server that does not run in the same k8s cluster

Based on the CRD this is supported however when it is deployed in the cluster this is reported back as we are using http://vault.vault.svc.cluster.local:8200/ just to test it the values are supported:

│ Events:                                                                                                                                                                         │
│   Type     Reason           Age                  From         Message                                                                                                           │
│   ----     ------           ----                 ----         -------                                                                                                           │
│   Warning  ProcessingError  94s (x17 over 7m2s)  VaultSecret  Error making API request.                                                                                         │
│                                                                                                                                                                                 │
│ URL: PUT http://vault.vault.svc.cluster.local:8200/v1/auth/kubernetes/login                                                                                                     │
│ Code: 400. Errors:                                                                                                                                                              │
│                                                                                                                                                                                 │
│ * missing role

Having looked into apiextensions.k8s.io_v1_customresourcedefinition_vaultsecrets.redhatcop.redhat.io.yaml there is no support for role when attempting to add role this is reported back: strict decoding error: unknown field "spec.vaultSecretDefinitions[0].connection.role"

When removing

 - connection:
    address: http://vault.vault.svc.cluster.local:8200/

This works as before but only to the vault server that happens to run on the same cluster as vault config operator is deployed into, plus role is defined under spec:vaultSecretDefinitions:authentication:role 

apiVersion: redhatcop.redhat.io/v1alpha1
kind: VaultSecret
metadata:
 namespace: helloworld
 name: helloworld-pull-secret
spec:
 vaultSecretDefinitions:
 - connection:
    address: http://vault.vault.svc.cluster.local:8200/
 - authentication:
     namespace: vault-config-authentications
     path: kubernetes
     role: kubernetes-auth-engine-admin-role
   name: retrievepassword
   path: teams/kv-test/data/cluster/helloworld/randomsecret-password-v1
 output:
   name: pull-secret-from-vault
   stringData:
    password : "{{ .retrievepassword.password }}"
   type: Opaque
   labels:
    app: helloworld
   annotations:
    purpose: spike-testing

Log output from Vault Config Operator (some things have been removed/redacted/replaced for obvious reasons)

2024-01-31T16:52:31.276861245Z 2024-01-31T16:52:31Z    ERROR    controllers.VaultSecret    unable to complete sync logic    {"instance": {"kind":"VaultSecret","apiVersion":"redhatcop.redhat.io/v1alpha1","metadata":{"name":"helloworld-pu
ll-secret","namespace":"helloworld","uid":"076fd7c0-12c2-47b9-b224-e2a00a7ee3a0","resourceVersion":"792013","generation":1,"creationTimestamp":"2024-01-31T16:52:10Z","annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"
redhatcop.redhat.io/v1alpha1\",\"kind\":\"VaultSecret\",\"metadata\":{\"annotations\":{},\"name\":\"helloworld-pull-secret\",\"namespace\":\"helloworld\"},\"spec\":{\"output\":{\"annotations\":{\"purpose\":\"spike-test
ing\"},\"labels\":{\"app\":\"helloworld\"},\"name\":\"pull-secret-from-vault\",\"stringData\":{\"password\":\"{{ .retrievepassword.password }}\"},\"type\":\"Opaque\"},\"refreshPeriod\":\"2m0s\",\"vaultSecretDefinitions\":[{\"connection\
":{\"address\":\"http://vault.vault.svc.cluster.local:8200/\"}},{\"authentication\":{\"namespace\":\"vault-config-authentications\",\"path\":\"kubernetes\",\"role\":\"kubernetes-auth-engine-admin-role\"},\"name\":\"retrievepassword\",\"
path\":\"teams/kv-test/data/cluster/helloworld/randomsecret-password-v1\"}]}}\n"},"managedFields":[{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"redhatcop.redhat.io/v1alpha1","time":"2024-01-31T16:52:10Z","fiel
dsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:output":{".":{},"f:annotations":{".":{},"f:purpose":{}},"f:labels":{".":{},"f:app":{}},"f:name":{},"f:st
ringData":{".":{},"f:password":{}},"f:type":{}},"f:refreshPeriod":{},"f:refreshThreshold":{},"f:vaultSecretDefinitions":{}}}},{"manager":"Go-http-client","operation":"Update","apiVersion":"redhatcop.redhat.io/v1alpha1","time":"2024-01-31T16:52:21Z","f
ieldsType":"FieldsV1","fieldsV1":{"f:status":{".":{},"f:conditions":{".":{},"k:{\"type\":\"ReconcileFailed\"}":{".":{},"f:lastTransitionTime":{},"f:message":{},"f:observedGeneration":{},"f:reason":{},"f:status":{},"f:type":{}}}}},"subresource":"status
"}]},"spec":{"refreshPeriod":"2m0s","refreshThreshold":90,"vaultSecretDefinitions":[{"connection":{"address":"http://vault.vault.svc.cluster.local:8200/"},"authentication":{"serviceAccount":{"name":"default"},"path":"kubernetes"},"path":"kubernetes","
requestType":"GET"},{"name":"retrievepassword","authentication":{"serviceAccount":{"name":"default"},"path":"kubernetes","role":"kubernetes-auth-engine-admin-role","namespace":"vault-config-authentications"},"path":"teams/kv-test/d
ata/cluster/helloworld/randomsecret-password-v1","requestType":"GET"}],"output":{"name":"pull-secret-from-vault","type":"Opaque","stringData":{"password":"{{ .retrievepassword.password }}"},"labels":{"app":"helloworld"},"annotat
ions":{"purpose":"spike-testing"}}},"status":{"conditions":[{"type":"ReconcileFailed","status":"False","observedGeneration":1,"lastTransitionTime":"2024-01-31T16:52:21Z","reason":"LastReconcileCycleFailed","message":"Error making API
 request.\n\nURL: PUT http://vault.vault.svc.cluster.local:8200/v1/auth/kubernetes/login\nCode: 400. Errors:\n\n* missing role"}]}}, "error": "Error making API request.\n\nURL: PUT http://vault.vault.svc.cluster.local:8200/v1/auth/kubernetes/login\nCo
de: 400. Errors:\n\n* missing role"}
2024-01-31T16:52:31.276872676Z github.com/redhat-cop/vault-config-operator/controllers.(*VaultSecretReconciler).Reconcile
2024-01-31T16:52:31.276877062Z     /home/runner/work/vault-config-operator/vault-config-operator/controllers/vaultsecret_controller.go:120
2024-01-31T16:52:31.276881209Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
2024-01-31T16:52:31.276885392Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:118
2024-01-31T16:52:31.276968696Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
2024-01-31T16:52:31.276982441Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:314
2024-01-31T16:52:31.276993472Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
2024-01-31T16:52:31.276998063Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265
2024-01-31T16:52:31.277002090Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
2024-01-31T16:52:31.277005923Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226
2024-01-31T16:52:31.277237955Z 2024-01-31T16:52:31Z    DEBUG    events    Error making API request.
2024-01-31T16:52:31.277250727Z
2024-01-31T16:52:31.277255663Z URL: PUT http://vault.vault.svc.cluster.local:8200/v1/auth/kubernetes/login
2024-01-31T16:52:31.277260084Z Code: 400. Errors:
2024-01-31T16:52:31.277264103Z
2024-01-31T16:52:31.277268910Z * missing role    {"type": "Warning", "object": {"kind":"VaultSecret","namespace":"helloworld","name":"helloworld-pull-secret","uid":"076fd7c0-12c2-47b9-b224-e2a00a7ee3a0","apiVersion":"redhatcop.redhat.io
/v1alpha1","resourceVersion":"792013"}, "reason": "ProcessingError"}
2024-01-31T16:52:31.287843261Z 2024-01-31T16:52:31Z    ERROR    Reconciler error    {"controller": "vaultsecret", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "VaultSecret", "VaultSecret": {"name":"helloworld-pull-secret"
,"namespace":"helloworld"}, "namespace": "helloworld", "name": "helloworld-pull-secret", "reconcileID": "129564ba-69f6-43b0-80b0-ec130ac6629b", "error": "Error making API request.\n\nURL: PUT http://vault.vault.svc.cluster.local:8200/v1
/auth/kubernetes/login\nCode: 400. Errors:\n\n* missing role"}
2024-01-31T16:52:31.288569528Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
2024-01-31T16:52:31.288586261Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324
2024-01-31T16:52:31.288593078Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
2024-01-31T16:52:31.288597778Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265
2024-01-31T16:52:31.288602203Z sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
2024-01-31T16:52:31.288606641Z     /home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226
trevorbox commented 7 months ago

Your example has the connection as a separate element. The connection should be in the entry with all other required elements to authenticate.

for example:

apiVersion: redhatcop.redhat.io/v1alpha1
kind: VaultSecret
metadata:
  namespace: helloworld
  name: helloworld-pull-secret
spec:
  vaultSecretDefinitions:
    - connection:
        address: http://vault.vault.svc.cluster.local:8200/
      authentication:
        namespace: vault-config-authentications
        path: kubernetes
        role: kubernetes-auth-engine-admin-role
      name: retrievepassword
      path: teams/kv-test/data/cluster/helloworld/randomsecret-password-v1
jimsmith commented 7 months ago

Thank you @trevorbox that worked 👍🏻

And thank you for the updated example, I am able to see the elements clearly now, looking into the crd I wasn't quite able to make out the flow of the elements just quite right.

👍🏻