[BUG] Advanced StatefulSet does not work

[hantmac]

What happened:

I use the belows to creates a guestbook application using advanced statefulset.

kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/docs/tutorial/v1/guestbook-statefulset.yaml
kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/docs/tutorial/v1/guestbook-service.yaml

But the asts does not generate pod:

What you expected to happen:

The asts work well.

How to reproduce it (as minimally and precisely as possible):

kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/docs/tutorial/v1/guestbook-statefulset.yaml
kubectl apply -f https://raw.githubusercontent.com/kruiseio/kruise/master/docs/tutorial/v1/guestbook-service.yaml

Then kubectl get asts to check.

Anything else we need to know?:

BTW, according the definition of Advanced StatefulSet , the short name sts and asts both work, but I can only use asts the sts for Advanced StatefulSet not work.

group: apps.kruise.io
  names:
    kind: StatefulSet
    listKind: StatefulSetList
    plural: statefulsets
    shortNames:
    - sts
    - asts
    singular: statefulset

Environment:

• Kruise version: v0.7.0
• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-19T08:38:20Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.9-aliyun.1", GitCommit:"7fc1645", GitTreeState:"", BuildDate:"2021-01-27T03:42:30Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

[FillZpp]

@hantmac please paste the print of kubectl get asts xxx -o yaml, see if any error message in status

[FillZpp]

BTW, according the definition of Advanced StatefulSet , the short name sts and asts both work, but I can only use asts the sts for Advanced StatefulSet not work.

sts is declared by the built-in statefulset and advanced statefulset, the former one has higher priority. So you have to use asts or sts.apps.kruise.io.

[hantmac]

@hantmac please paste the print of kubectl get asts xxx -o yaml, see if any error message in status

apiVersion: apps.kruise.io/v1beta1
kind: StatefulSet
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apps.kruise.io/v1beta1","kind":"StatefulSet","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"demo-v1","app.kubernetes.io/name":"guestbook-kruise"},"name":"demo-v1-guestbook-kruise","namespace":"default"},"spec":{"replicas":2,"selector":{"matchLabels":{"app.kubernetes.io/instance":"demo-v1","app.kubernetes.io/name":"guestbook-kruise"}},"serviceName":"demo-v1-guestbook-kruise","template":{"metadata":{"labels":{"app.kubernetes.io/instance":"demo-v1","app.kubernetes.io/name":"guestbook-kruise"}},"spec":{"containers":[{"image":"openkruise/guestbook:v1","imagePullPolicy":"Always","name":"guestbook-kruise","ports":[{"containerPort":3000,"name":"http-server"}]}]}}}}
  creationTimestamp: "2021-05-21T02:57:20Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: demo-v1
    app.kubernetes.io/name: guestbook-kruise
  name: demo-v1-guestbook-kruise
  namespace: default
  resourceVersion: "2852576838"
  selfLink: /apis/apps.kruise.io/v1beta1/namespaces/default/statefulsets/demo-v1-guestbook-kruise
  uid: d1b93f80-ba8d-4d71-85db-a318a7fb3062
spec:
  podManagementPolicy: Parallel
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: demo-v1
      app.kubernetes.io/name: guestbook-kruise
  serviceName: demo-v1-guestbook-kruise
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: demo-v1
        app.kubernetes.io/name: guestbook-kruise
    spec:
      containers:
      - image: openkruise/guestbook:v1
        imagePullPolicy: Always
        name: guestbook-kruise
        ports:
        - containerPort: 3000
          name: http-server
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      readinessGates:
      - conditionType: InPlaceUpdateReady
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
  updateStrategy:
    rollingUpdate:
      maxUnavailable: 3
      minReadySeconds: 0
      partition: 0
      podUpdatePolicy: InPlaceIfPossible
    type: RollingUpdate

[FillZpp]

If there is no status in the yaml, maybe the kruise-controller-manager is in some abnormal states. You may check the state and log of it.

[hantmac]

The kruise-controller-manager seems in normal state :

And some recent logs about asts of leader controller :

I0521 03:49:31.902181       1 http.go:84] controller-runtime/webhook/webhooks "msg"="received request" "webhook"="/mutate-apps-kruise-io-statefulset" "UID"="272c2dab-ca84-470d-a4a0-13aecc69c4f7" "kind"={"group":"apps.kruise.io","version":"v1beta1","kind":"StatefulSet"} "resource"={"group":"apps.kruise.io","version":"v1beta1","resource":"statefulsets"}
I0521 03:49:31.905353       1 http.go:84] controller-runtime/webhook/webhooks "msg"="received request" "webhook"="/validate-apps-kruise-io-statefulset" "UID"="0ca06dd9-c932-4470-b66f-451b5c115f62" "kind"={"group":"apps.kruise.io","version":"v1beta1","kind":"StatefulSet"} "resource"={"group":"apps.kruise.io","version":"v1beta1","resource":"statefulsets"}

Some recent logs of leader controller:

I0521 03:53:04.836331       1 webhook_controller.go:232] Finished to sync webhook certs and configurations
I0521 03:53:37.791222       1 reflector.go:383] pkg/webhook/util/controller/webhook_controller.go:194: Watch close - *v1beta1.CustomResourceDefinition total 0 items received
I0521 03:54:01.922612       1 reflector.go:383] sigs.k8s.io/controller-runtime/pkg/cache/internal/informers_map.go:136: Watch close - *v1.PersistentVolumeClaim total 0 items received
I0521 03:54:04.850860       1 webhook_controller.go:230] Starting to sync webhook certs and configurations
I0521 03:54:04.855476       1 atomic_writer.go:330] current pathstarget directory/tmp/kruise-webhook-certspaths[ca-cert.pem ca-key.pem cert.pem key.pem tls.crt tls.key]
I0521 03:54:04.855500       1 atomic_writer.go:342] new pathstarget directory/tmp/kruise-webhook-certspaths[ca-cert.pem ca-key.pem cert.pem key.pem tls.crt tls.key]
I0521 03:54:04.855512       1 atomic_writer.go:345] paths to removetarget directory/tmp/kruise-webhook-certspathsmap[]
I0521 03:54:04.855640       1 atomic_writer.go:155] no update required for target directorydirectory/tmp/kruise-webhook-certs
W0521 03:54:04.877549       1 configuration.go:66] Find path /mutate-pod not in handlers map[/mutate-apps-kruise-io-statefulset:0xc00040e7f8 /mutate-apps-kruise-io-v1alpha1-advancedcronjob:0xc0002084d0 /mutate-apps-kruise-io-v1alpha1-broadcastjob:0xc000208540 /mutate-apps-kruise-io-v1alpha1-cloneset:0xc00040e7b8 /mutate-apps-kruise-io-v1alpha1-daemonset:0xc00040e7d0 /mutate-apps-kruise-io-v1alpha1-sidecarset:0xc00040e7e0 /mutate-apps-kruise-io-v1alpha1-uniteddeployment:0xc00040e810 /validate-apps-kruise-io-statefulset:0xc00040e808 /validate-apps-kruise-io-v1alpha1-advancedcronjob:0xc000208538 /validate-apps-kruise-io-v1alpha1-broadcastjob:0xc00040e7b0 /validate-apps-kruise-io-v1alpha1-cloneset:0xc00038db00 /validate-apps-kruise-io-v1alpha1-daemonset:0xc00040e7d8 /validate-apps-kruise-io-v1alpha1-sidecarset:0xc00040e7f0 /validate-apps-kruise-io-v1alpha1-uniteddeployment:0xc00040e820]
I0521 03:54:04.904321       1 webhook_controller.go:232] Finished to sync webhook certs and configurations

Any abnormal states?

[shiyan2016]

@hantmac Do you make a upgrade for kruise from some version to 0.8.1?

[FillZpp]

grep the demo-v1-guestbook-kruise name in the logs, see if any warning?

[hantmac]

grep the demo-v1-guestbook-kruise name in the logs, see if any warning?

Nothing about demo-v1-guestbook-kruise.

[hantmac]

@hantmac Do you make a upgrade for kruise from some version to 0.8.1?

I am in version v0.7.0

[hantmac]

Recreating kruise controller does not work but I upgrade version to v0.8.1 , it works.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.