Custom service account

[ernoaapa]

At the moment when vault-operator creates pod, it uses the default service account. I would like to define custom service account so I can, for example, define Pod Security Policies only for the vault pod.

I can probably provide PR of this but would like to get some feedback first. Is there something that I should remember?

[arthur0]

I hope to help with my example:

#  I'm using a namespace called vault-system 
apiVersion: v1
kind: Namespace
metadata:
  name: vault-system
---
# Service Account for the etcd-operator
apiVersion: v1
kind: ServiceAccount
metadata:
  name: etcd-operator
  namespace: vault-system
---
# Role for the etcd-operator
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: etcd-operator
  namespace: vault-system
rules:
- apiGroups:
  - etcd.database.coreos.com
  resources:
  - etcdclusters
  - etcdbackups
  - etcdrestores
  verbs:
  - "*"
- apiGroups:
  - apiextensions.k8s.io
  resources:
  - customresourcedefinitions
  verbs:
  - "*"
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  verbs:
  - "*"
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - "*"
# The following permissions can be removed if not using S3 backup and TLS
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
---
# Role Binding for the etcd-operator
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: etcd-operator
  namespace: vault-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: etcd-operator
subjects:
- kind: ServiceAccount
  name: etcd-operator
  namespace: vault-system
---
# Service Account for the vault-operator
apiVersion: v1
kind: ServiceAccount
metadata:
  name: vault-operator
  namespace: vault-system
---
# Role for the vault-operator
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: vault-operator
  namespace: vault-system
rules:
- apiGroups:
  - etcd.database.coreos.com
  resources:
  - etcdclusters
  - etcdbackups
  - etcdrestores
  verbs:
  - "*"
- apiGroups:
  - vault.security.coreos.com
  resources:
  - vaultservices
  verbs:
  - "*"
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  verbs:
  - "*"
- apiGroups:
  - "" # "" indicates the core API group
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  - secrets
  verbs:
  - "*"
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - "*"
---
# Role Binding for the vault-operator
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: vault-operator
  namespace: vault-system
subjects:
- kind: ServiceAccount
  name: vault-operator
  namespace: vault-system
roleRef:
  kind: Role
  name: vault-operator
  apiGroup: rbac.authorization.k8s.io

So, the Service Account for the etcd and vault operators must be specified in the deployments manifests to produce this output:

$ kubectl get deploy -n vault-system vault-operator -o jsonpath={".spec.template.spec.serviceAccountName"}
vault-operator

$ kubectl get deploy -n vault-system etcd-operator -o jsonpath={".spec.template.spec.serviceAccountName"}
etcd-operator

[ernoaapa]

What I meant, was that when etcd-operator creates vault pods, the vault pods uses default ServiceAccount. I'm looking for way to set the ServiceAccount for the vault pods. I.e. set the serviceAccountName for the pod in here https://github.com/coreos/vault-operator/blob/master/pkg/util/k8sutil/vault.go#L239-L257

[rblaine95]

I'm also looking for a way to do this as I don't believe it's very wise to give the default service account the required privileges for vault.

I prefer keeping the default service account as a restricted security context constraint, even in the vault namespace.

To deploy the Vault pods, the service account the pods are running as needs to have the nonroot security context constraint (SCC), as well as being able to set the IPC_LOCK capability. Custom capability settings is only in the privileged SCC.

The solution would be to create a custom SCC, merging nonroot with the allowedCapabilities key set to all, to maintain a whitelist or minimum required permissions setup.

allowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegedContainer: false
allowedCapabilities:
- '*'
allowedFlexVolumes: null
apiVersion: security.openshift.io/v1
defaultAddCapabilities: null
fsGroup:
  type: RunAsAny
groups: []
kind: SecurityContextConstraints
metadata:
  annotations:
    kubernetes.io/description: vault scc provides all features of the restricted SCC
      but allows users to run with any non-root UID, as well as specify Capabilities such as IPC_LOCK.
      UIDs must be specified by the user when running pods with this SCC.
  name: vault-scc
priority: null
readOnlyRootFilesystem: false
requiredDropCapabilities: null
runAsUser:
  type: MustRunAsNonRoot
seLinuxContext:
  type: MustRunAs
supplementalGroups:
  type: RunAsAny
users:
- system:serviceaccount:vaultops:vault-sa
- system:serviceaccount:vaultops:etcd-operator
- system:serviceaccount:vaultops:vault-operator
volumes:
- configMap
- downwardAPI
- emptyDir
- persistentVolumeClaim
- projected
- secret

Edit: What would be nice is to be able to specify a custom service account in the vault service manifest. For example:

apiVersion: "vault.security.coreos.com/v1alpha1"
kind: "VaultService"
metadata:
  name: "vault"
spec:
  nodes: 3
  version: "0.9.1-0"
  serviceAccount: vault-sa

Edit: Perhaps this could be a solution? I'm not a Go developer and I'm not entirely certain how correct this is

// pkg/apis/vault/v1alpha1/typs.go

type VaultServiceSpec struct {
/*
rest of struct snipped
*/
ServiceAccountName string `json:"serviceAccountName,omitempty"`
}

// util/k8sutil/vault.go
func DeployVault(kubecli kubernetes.Interface, v *api.VaultService) error {
    selector := LabelsForVault(v.GetName())

    podTempl := v1.PodTemplateSpec{
        ObjectMeta: metav1.ObjectMeta{
            Name:   v.GetName(),
            Labels: selector,
        },
        Spec: v1.PodSpec{
            ServiceAccountName: v.Spec.ServiceAccountName,
            Containers: []v1.Container{vaultContainer(v), statsdExporterContainer()},
            Volumes: []v1.Volume{{
                Name: vaultConfigVolName,
                VolumeSource: v1.VolumeSource{
                    ConfigMap: &v1.ConfigMapVolumeSource{
                        LocalObjectReference: v1.LocalObjectReference{
                            Name: ConfigMapNameForVault(v),
                        },
                    },
                },
            }},
        },
    }
/*
rest of function snipped
*/