Pos status: `Failed` does not trigger the webhook

[izhangzhihao]

to reproduce:

kubectl apply -f https://gist.githubusercontent.com/izhangzhihao/cb7e653a71b4dc1b8dc34237d5ce43b8/raw/96f141bf3a3ddc13d41842ff17b9d2207eb49488/fail.yml

the content of the yml is

apiVersion: v1
kind: Pod
metadata:
  name: fail
spec:
  restartPolicy: Never
  containers:
    - name: python
      image: python:3.7-alpine3.12
      command: ["/bin/sh"]
      args:
        - python
        - -c
        - "exit(-1)"

So it will just fail, and after that, you can see

kubectl describe pod fail

and the result is

Name:         fail
Namespace:    xxxx
Priority:     0
Node:         cn-shanghai.xxxxxxx
Start Time:   Wed, 30 Dec 2020 20:08:09 +0800
Labels:       <none>
Annotations:  <none>
Status:       Failed
IP:           xxxx
IPs:
  IP:  xxxx
Containers:
  python:
    Container ID:  docker://f739e325eb126f840519a773b6e8013cf0c40005c4aa5d0addb4ad70f572d2c9
    Image:         python:3.7-alpine3.12
    Image ID:      docker-pullable://python@sha256:a73d0fdab3235920c0df44d55c420e2c3096f1b1a95e5f913a0287eee4192bdb
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
    Args:
      python
      -c
      exit(-1)
    State:          Terminated
      Reason:       Error
      Exit Code:    2
      Started:      Wed, 30 Dec 2020 20:08:10 +0800
      Finished:     Wed, 30 Dec 2020 20:08:10 +0800
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-bktgd (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  default-token-bktgd:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-bktgd
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  55s   default-scheduler  Successfully assigned aid-recomm-dev/fail to cn-shanghai.192.168.0.242
  Normal  Pulled     55s   kubelet            Container image "python:3.7-alpine3.12" already present on machine
  Normal  Created    55s   kubelet            Created container python
  Normal  Started    55s   kubelet            Started container python

And the Status of the Pod is Failed, but no events comes from kube-eventer:

[ringtail]

 Normal  Scheduled  55s   default-scheduler  Successfully assigned aid-recomm-dev/fail to cn-shanghai.192.168.0.242
  Normal  Pulled     55s   kubelet            Container image "python:3.7-alpine3.12" already present on machine
  Normal  Created    55s   kubelet            Created container python
  Normal  Started    55s   kubelet            Started container python

Here are the events. There are no failed events above.

[izhangzhihao]

Anyone run into this problem can try this to invoke an unhealthy event:

apiVersion: v1
kind: Pod
metadata:
  name: healthy
spec:
  restartPolicy: Never
  containers:
    - name: python
      image: python:3.7-alpine3.12
      args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
      livenessProbe:
        exec:
          command:
            - cat
            - /tmp/healthy