preStop hook調査

[oomichi]

ドキュメント にはpreStop hookの挙動について以下のように記載されているが、

Note: Kubernetes only sends the preStop event when a Pod is terminated.
This means that the preStop hook is not invoked when the Pod is completed.

LivenessProve/Startup失敗時にpod内のコンテナが終了した際にもpreStop hookは呼ばれるため、実際の挙動と異なる。 "when a Pod is terminated"とは少し異なると考えられる。 したがって、ドキュメントを下記のとおり修正して欲しい。

Kubernetes only sends the preStop event when a Pod or a container in the Pod is terminated.
This means that the preStop hook is not invoked when the Pod is completed.

[oomichi]

上記の投稿内容に対し、ローカル環境で以下の手順で動作確認を実施しました。 以下のyamlファイルを作成（liveness-check.yaml）

apiVersion: v1
kind: Pod
metadata:
  name: liveness-check
spec:
  containers:
  - image: nginx
    name: nginx
    livenessProbe:
      httpGet:
        port: 80
        path: /
      failureThreshold: 5
      periodSeconds: 5
    lifecycle:
      preStop:
        exec:
          command: ["/usr/sbin/nginx","-s","quit"]

2. 上記のyamlファイルからpodを作成

   $ kubectl apply -f liveness-check.yaml

3. /usr/share/nginx/html/index.html を削除する

   $ kubectl exec liveness-check -- rm /usr/share/nginx/html/index.html

4. podの詳細を確認する

   $ kubectl describe pod liveness-check
   ...
   Events:
   Type     Reason     Age                    From               Message
   ----     ------     ----                   ----               -------
   Normal   Scheduled  7m34s                  default-scheduler  Successfully assigned default/liveness-check to test-worker
   Normal   Pulled     7m30s                  kubelet            Successfully pulled image "nginx" in 2.291357524s
   Warning  Unhealthy  6m29s (x5 over 6m49s)  kubelet            Liveness probe failed: HTTP probe failed with statuscode: 403
   Normal   Killing    6m29s                  kubelet            Container nginx failed liveness probe, will be restarted
   Normal   Pulling    6m28s (x2 over 7m32s)  kubelet            Pulling image "nginx" 
   Normal   Pulled     6m26s                  kubelet            Successfully pulled image "nginx" in 2.119307294s
   Normal   Created    6m24s (x2 over 7m29s)  kubelet            Created container nginx
   Normal   Started    6m24s (x2 over 7m28s)  kubelet            Started container nginx

[oomichi]

preStopイベントとは？

it sends the preStop event immediately before the Container is terminated.

[oomichi]

preStopがlivenessProbeで動作することを確認

pod を作成する

$ cat pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: nginx
    livenessProbe:
      httpGet:
        port: 80
        path: /
      failureThreshold: 5
      periodSeconds: 5
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler >> /var/local/aaa/message"]
      preStop:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the preStop handler >> /var/local/aaa//message"]
    volumeMounts:
    - mountPath: /var/local/aaa
      name: mydir
  volumes:
  - name: mydir
    hostPath:
      # Ensure the file directory is created.
      path: /var/local/aaa
      type: DirectoryOrCreate
$ kubectl apply -f pod.yaml

index.html を削除して、livenessProbeが動作するようにする

$ kubectl exec lifecycle-demo -- rm /usr/share/nginx/html/index.html
$ kubectl exec lifecycle-demo -- ls /usr/share/nginx/html/
50x.html

restart したことを検知、ログに preStop が発生したことが記録されている

$ kubectl get pods
NAME             READY   STATUS    RESTARTS     AGE
lifecycle-demo   1/1     Running   1 (3s ago)   78s
$ kubectl exec lifecycle-demo -- cat /var/local/aaa/message
Hello from the postStart handler
Hello from the preStop handler
Hello from the postStart handler
$