search

hashicorp / vault-helm

Helm chart to install Vault and other associated components.
Mozilla Public License 2.0
1.05k stars 868 forks source link

An error occurred in the init container(Operation not permitted) #973

Open sdYoo opened 8 months ago

sdYoo commented 8 months ago

Describe the bug After executing the helm chart, the following error occurs when running the init container.

➜ ~ kc logs -f vault-0 -c busybox -n security chown: /vault/logs: Operation not permitted chown: /vault/logs: Operation not permitted

To Reproduce Steps to reproduce the behavior:

  1. Install chart
  2. Run vault command
  3. See error (vault logs, etc.)

vault-0 0/1 Init:CrashLoopBackOff 5 (60s ago) 4m13s 10.252.6.12 node01 vault-1 0/1 Init:CrashLoopBackOff 5 (65s ago) 4m11s 10.252.26.87 node02

Other useful info to include: vault pod logs, kubectl describe statefulset vault and kubectl get statefulset vault -o yaml output

Name: vault Namespace: security CreationTimestamp: Mon, 30 Oct 2023 16:41:15 +0900 Selector: app.kubernetes.io/instance=vault,app.kubernetes.io/name=vault,component=server Labels: app.kubernetes.io/instance=production-retail-mgmt-security-vault app.kubernetes.io/managed-by=Helm app.kubernetes.io/name=vault Annotations: Replicas: 2 desired | 2 total Update Strategy: OnDelete Pods Status: 0 Running / 2 Waiting / 0 Succeeded / 0 Failed Pod Template: Labels: app.kubernetes.io/instance=vault app.kubernetes.io/name=vault component=server helm.sh/chart=vault-0.24.1 Annotations: kubectl.kubernetes.io/restartedAt: 2023-11-06T08:02:25Z Service Account: vault Init Containers: busybox: Image: docker-hub.com/finalspy/busybox-curl-jq Port: Host Port: Command: sh -c chown -R 1000:1000 /vault/logs Environment: Mounts: /vault/logs from logs (rw) Containers: vault: Image: docker-hub.com/hashicorp/vault:1.13.1-jqcurl Ports: 8200/TCP, 8201/TCP, 8202/TCP Host Ports: 0/TCP, 0/TCP, 0/TCP Command: /bin/sh -ec Args: cp /vault/config/extraconfig-from-values.hcl /tmp/storageconfig.hcl; [ -n "${HOST_IP}" ] && sed -Ei "s|HOST_IP|${HOST_IP?}|g" /tmp/storageconfig.hcl; [ -n "${POD_IP}" ] && sed -Ei "s|POD_IP|${POD_IP?}|g" /tmp/storageconfig.hcl; [ -n "${HOSTNAME}" ] && sed -Ei "s|HOSTNAME|${HOSTNAME?}|g" /tmp/storageconfig.hcl; [ -n "${API_ADDR}" ] && sed -Ei "s|API_ADDR|${API_ADDR?}|g" /tmp/storageconfig.hcl; [ -n "${TRANSIT_ADDR}" ] && sed -Ei "s|TRANSIT_ADDR|${TRANSIT_ADDR?}|g" /tmp/storageconfig.hcl; [ -n "${RAFT_ADDR}" ] && sed -Ei "s|RAFT_ADDR|${RAFT_ADDR?}|g" /tmp/storageconfig.hcl; /usr/local/bin/docker-entrypoint.sh vault server -config=/tmp/storageconfig.hcl

Limits:
  cpu:     250m
  memory:  256Mi
Requests:
  cpu:     250m
  memory:  256Mi
Environment:
  HOST_IP:               (v1:status.hostIP)
  POD_IP:                (v1:status.podIP)
  VAULT_K8S_POD_NAME:    (v1:metadata.name)
  VAULT_K8S_NAMESPACE:   (v1:metadata.namespace)
  VAULT_ADDR:           http://127.0.0.1:8200
  VAULT_API_ADDR:       http://$(POD_IP):8200
  SKIP_CHOWN:           true
  SKIP_SETCAP:          true
  HOSTNAME:              (v1:metadata.name)
  VAULT_CLUSTER_ADDR:   https://$(HOSTNAME).vault-internal:8201
  HOME:                 /home/vault
  VAULT_LOG_LEVEL:      info
  VAULT_LOG_FORMAT:     json
Mounts:
  /etc/localtime from localtime (ro)
  /etc/timezone from timezone (ro)
  /home/vault from home (rw)
  /vault/config from config (rw)
  /vault/file from script (rw)
  /vault/logs from logs (rw)

Volumes: config: Type: ConfigMap (a volume populated by a ConfigMap) Name: vault-config Optional: false logs: Type: HostPath (bare host directory volume) Path: /home/logs/security/vault HostPathType: home: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: timezone: Type: HostPath (bare host directory volume) Path: /etc/timezone HostPathType: localtime: Type: HostPath (bare host directory volume) Path: /etc/localtime HostPathType: script: Type: ConfigMap (a volume populated by a ConfigMap) Name: vault-auto-unseal Optional: false Volume Claims: Events: Type Reason Age From Message

Normal SuccessfulCreate 8m6s (x24 over 7d3h) statefulset-controller create Pod vault-0 in StatefulSet vault successful Normal SuccessfulCreate 8m4s (x24 over 7d3h) statefulset-controller create Pod vault-1 in StatefulSet vault successful

Expected behavior A clear and concise description of what you expected to happen.

Environment

Chart values:


values.yaml

server:
  # Configure the logging format for the Vault server.
  # Supported log formats include: standard, json
  logFormat: "json"

  extraInitContainers:
  - name: busybox
    image: "docker-hub-custom.com/finalspy/busybox-curl-jq"
    command: [ "sh", "-c", "chown -R 1000:1000 /vault/logs" ]
    volumeMounts:
      - name: logs
        mountPath: /vault/logs

Additional context Add any other context about the problem here.

sdYoo commented 8 months ago

apiVersion: apps/v1 kind: StatefulSet metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"apps/v1","kind":"StatefulSet","metadata":{"annotations":{},"labels": --------------- creationTimestamp: "2023-10-30T07:41:15Z" generation: 27 labels: app.kubernetes.io/instance: security-vault app.kubernetes.io/managed-by: Helm app.kubernetes.io/name: vault name: vault namespace: security resourceVersion: "49245057" uid: a08381cc-4a96-4543-90cd-dd8eeb79f22b spec: podManagementPolicy: Parallel replicas: 2 revisionHistoryLimit: 10 selector: matchLabels: app.kubernetes.io/instance: vault app.kubernetes.io/name: vault component: server serviceName: vault-internal template: metadata: annotations: kubectl.kubernetes.io/restartedAt: "2023-11-06T08:02:25Z" creationTimestamp: null labels: app.kubernetes.io/instance: vault app.kubernetes.io/name: vault component: server helm.sh/chart: vault-0.24.1 spec: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution:

sdYoo commented 8 months ago

I solved the permission issue, but an error occurs as below and the pod is pending.

=========== state: terminated: containerID: containerd://484566bf23bc63fe7f3f0c9474dd52226da130fd29137ee3f095a92569b17fe9 exitCode: 0 finishedAt: "2023-11-08T01:04:55Z" reason: Completed startedAt: "2023-11-08T01:04:55Z" phase: Pending podIP: 10.252.23.241 podIPs: