Closed njohnstone2 closed 1 year ago
Sounds like it's similarly adjacent to the problem posed in #128 where you want to distribute a standard secret across all namespaces. Is that the intended goal? Or do you just want a dedicated namespace to handle distributing secrets to other namespaces?
In a similar vein of lowering overhead to configure a new k8s namespace, I could see following the Cluster*
pattern (i.e., ClusterRoleBinding
vs Rolebinding
, ClusterTask
vs Task
in Tekton) for resources like VaultConnection
so it can be configured once and referenced by any kube namespace in the cluster.
Or do you just want a dedicated namespace to handle distributing secrets to other namespaces?
That's the usecase I have in mind at least. I'd like to deploy the operator to the namespace containing my Vault ctluster, and have it be responsible for creating different secrets in different namespaces.
This also seems to follow from the defaultVaultConnection
and defaultAuthMethod
values in the Helm chart. Specify a default, and then that's responsible for mananging all secrets across the cluster.
Or do you just want a dedicated namespace to handle distributing secrets to other namespaces?
This is my use case as well. Like @ameyp I also planned to use the default connection/auth from the helm values to manage my vault secrets.
In a similar vein of lowering overhead to configure a new k8s namespace, I could see following the
Cluster*
pattern (i.e.,ClusterRoleBinding
vsRolebinding
,ClusterTask
vsTask
in Tekton) for resources likeVaultConnection
so it can be configured once and referenced by any kube namespace in the cluster.
I would be happy with this solution. I think this would require the VaultSecret
resource to exist in the destination namespace but that is fine for my use case.
Let me know if you disagree but I see two possible configuration patterns:
VaultSecret
resources defined in a single namespace and the ability to specify a destination
namespaceVaultSecret
resource in the target
kubernetes namespace that references the vaultAuthRef
in a separate namespaceUsing unique vaultAuthRef require to expose all your secret to the attached service account. and with this vaultAuth every namespace can request Secret all secret. for me using a unique vaultauth accross namespaces is a security breach.
@njohnstone2 @sebglon @njohnstone2 @TheLonelyGhost so for the moment, we can only deploy secret on 1 namespace? I'm using this link to try vault-secrets-operator after installing of the plugin using this tutos (https://developer.hashicorp.com/vault/tutorials/kubernetes/vault-secrets-operator#the-vault-secrets-operator) helm install vault-secrets-operator hashicorp/vault-secrets-operator --version 0.1.0-beta -n vault --values vault/vault-operator-values.yaml
defaultAuthMethod:
enabled: true
# Vault namespace for the VaultAuthMethod CR
namespace: "app"
method: kubernetes
mount: kubernetes
im only able to create secrets on the namespace app only, any idea how to add another namespace?
I'm currently working around the limitation by duplicating the 'VaultConnection' and 'VaultAuth' resources to each namespace that needs to sync secrets
thanks @njohnstone2 , so for the moment we can use the operator only for one namespace? we can't duplicate the 'VaultConnection' and 'VaultAuth' manually?
It is possible to use single default VaultAuth and VaultConnection created by Helm chart in the namespace where the operator is installed. To be able to use the default VaultAuth from different namespace, the "vaultAuthRef:" should not be used in VaultSecret CRDs as pointed out in case #148 . Caveat is that a Service Account with the same name (the name which is defined in values file while deploying via Helm) must be created in all namespaces where VaultSecret CRDs will be used. Example: Vault Secrets Operator is being installed "vault-test" namespace via Helm with below values:
defaultVaultConnection:
enabled: true
address: "https://x.x.x.x"
skipTLSVerify: true
defaultAuthMethod:
enabled: true
mount: dev-eks
kubernetes:
role: "dev-vault-secret-operator"
serviceAccount: vault-secrets-operator-controller-manager
Now by default Helm chart creates Service Account for the operator named "vault-secrets-operator-controller-manager" so if you want you can use same Service Account for defaultAuthMethod or you can create and use another one. In above values, the helm created Service Account is being used. The importance of this Service Account name is that , Service Account with the same name depicted in the values file must be created in all namespaces where VaultSecret CRD will be deployed. Note: Since K8s 1.24 Service Account + its secret + token reviewer cluster role binding must be created by user. After the SA is deployed in required namespaces then VaultSecret CRD can be deployed without specifying "vaultAuthRef":
---
apiVersion: secrets.hashicorp.com/v1alpha1
kind: VaultStaticSecret
metadata:
namespace: devops
name: example-devops-static
spec:
mount: sbx
type: kv-v1
name: toto-test
refreshAfter: 60s
destination:
create: true
name: toto-test-devops
Consclusion: example-devops-static VaultStaticSecret CRD will use VaultAuth default which is in Vault Namespace and default VaultAuth will look for the SA named vault-secrets-operator-controller-manager (or any other SA name which is defined in values file) in the Namespace where VaultStaticSecret CRD is deployed.
@togrulazizli - If you use the default service account like so:
---
defaultVaultConnection:
enabled: true
address: "http://vault.vault.svc.cluster.local:8200"
defaultAuthMethod:
enabled: true
kubernetes:
role: ${vault-kube-auth-role}
serviceAccount: default
tokenAudiences:
- vault
Then that already exists in every namespace.
The rest of my config is here: https://github.com/shutthegoatup/homelab and it appears to be working.
@allandegnan
Hey allan, i tried your method of deploying vso to a kubnenetes cluster that then connects back a external vault cluster, and the connections seems to be working fine, however when i try directly applying secrets such as:
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
namespace: demo
name: config
spec:
mount: secret
type: kv-v2
name: myapp/config
refreshAfter: 60s
destination:
create: true
name: test-secret
I would get errors such as:
error: resource mapping not found for name: "config" namespace: "demo" from "secret2.yaml": no matches for kind "VaultStaticSecret" in version "secrets.hashicorp.com/v1beta1"
ensure CRDs are installed first
should we deploy secrets in a different way? I am assuming that since we already have defaultAuthMethod:
block there is no need to re-apply something like the below since we are going through default service account auth.
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: static-auth
namespace: app
spec:
method: kubernetes
mount: test-dev
kubernetes:
role: vso
serviceAccount: default
but it seems like on my end that if i dont apply VaultAuth, when applying secrets even when getting Connection Accepted in the vso logs i would get ensure CRDs are installed first
@allandegnan
Hey allan, i tried your method of deploying vso to a kubnenetes cluster that then connects back a external vault cluster, and the connections seems to be working fine, however when i try directly applying secrets such as:
apiVersion: secrets.hashicorp.com/v1beta1 kind: VaultStaticSecret metadata: namespace: demo name: config spec: mount: secret type: kv-v2 name: myapp/config refreshAfter: 60s destination: create: true name: test-secret
I would get errors such as:
error: resource mapping not found for name: "config" namespace: "demo" from "secret2.yaml": no matches for kind "VaultStaticSecret" in version "secrets.hashicorp.com/v1beta1" ensure CRDs are installed first
should we deploy secrets in a different way? I am assuming that since we already have
defaultAuthMethod:
block there is no need to re-apply something like the below since we are going through default service account auth.apiVersion: secrets.hashicorp.com/v1beta1 kind: VaultAuth metadata: name: static-auth namespace: app spec: method: kubernetes mount: test-dev kubernetes: role: vso serviceAccount: default
but it seems like on my end that if i dont apply VaultAuth, when applying secrets even when getting Connection Accepted in the vso logs i would get
ensure CRDs are installed first
Assuming the CRD message is a red herring, I think your path is probably wrong.
My mount point is "kvv2" and my path is everything after that.
- apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultStaticSecret
metadata:
annotations:
meta.helm.sh/release-name: arc-github-app
meta.helm.sh/release-namespace: actions-runner-controller
creationTimestamp: "2023-07-02T13:42:32Z"
generation: 1
labels:
app: raw
app.kubernetes.io/managed-by: Helm
chart: raw-v0.3.2
heritage: Helm
release: arc-github-app
name: arc-github-app
namespace: actions-runner-controller
resourceVersion: "8008422"
uid: e1d23224-7df5-40a1-9e73-648e32cb2868
spec:
destination:
create: true
name: arc-github-app
hmacSecretData: true
mount: kvv2
path: arc-github-app
refreshAfter: 60s
type: kv-v2
Here's a screenshot of my vault https://imgur.com/a/7WZ7dta
Hope that helps.
I am surprised that this issue has been closed as completed. It's still not possible to use a single, global ServiceAccount for VaultStaticSecrets across all namespaces of the cluster.
Even when using this VaultAuth
, you still need a separate vault-default
ServiceAccount in each namespace:
apiVersion: secrets.hashicorp.com/v1beta1
kind: VaultAuth
metadata:
name: default
spec:
vaultConnectionRef: default
allowedNamespaces: ['*']
method: jwt
mount: jwt-external-services
jwt:
role: vso-default
serviceAccount: vault-default
audiences:
- vault # must match the `bound_audiences` of the vault role
EDIT: Please disregard that. While this is technically true, I just found out that bound_subject
of the role definition is an optional attribute. I can just use bound_audiences
and use the default
ServiceAccount that exists by default in every namespace.
Is your feature request related to a problem? Please describe. The output kubernetes secret from a sync operation is always stored in the namespace that contains the
VaultSecret*
custom resource.Theres two main problems with this:
Describe the solution you'd like It would be good if the Destination spec could support specifying the desired kubernetes namespace for the secret to be created in.
Describe alternatives you've considered Creating
VaultConnection
,VaultAuth
andVaultSecret*
custom resources all in the desired kubernetes namespace works as expected but this means that you end up with duplicate Connection/Auth custom resources spread across multiple kubernetes namespaces