kubelet貌似一直在重启，百思不得其解啊

i0668 commented 5 years ago

文档版本 v1.12。

现象描述 貌似kubelet一直在重启，导致csr一直在增加，大约每12分钟重启一次。我看了下kubelet的日志，有一句可能是关键： May 29 17:42:55 kube1 kubelet[4841]: E0529 17:42:55.179148 4841 certificate_manager.go:378] Certificate request was not signed: timed out waiting for the condition 每次抛出这个错误之后就重启了。

opsnull commented 5 years ago

请严格 follow 文档，执行这两个步骤后， CSR 应该会被签名的：

leiyazhen commented 5 years ago

请严格 follow 文档，执行这两个步骤后， CSR 应该会被签名的：

请教一下，我参照文档来的，3台机器，其中一台kubelet server启动正常后，另外一台通过手动approve后也正常，剩下一台执行完手动approve也能通过，但是就是启动失败，而且每次都自动签名approve很多个证书，查看错误定位到k8s源码是这里报错，卡在这几天了，能指点一下吗

// Start initializes and starts components in dockerService.
func (ds *dockerService) Start() error {
    ds.initCleanup()

    // Initialize the legacy cleanup flag.
    if ds.startLocalStreamingServer {
        go func() {
            if err := ds.streamingServer.Start(true); err != nil {### //报错位置
                klog.Fatalf("Streaming server stopped unexpectedly: %v", err)
            }
        }()
    }
    return ds.containerManager.Start()
}

jouralctl 日志

7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.219601   24658 server.go:997] Using root directory: /data/k8s/k8s/kubelet
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.219625   24658 kubelet.go:304] Watching apiserver
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.221087   24658 client.go:75] Connecting to docker on unix:///var/run/docke
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.221208   24658 client.go:104] Start docker client with request timeout=10m
7月 17 17:38:28 bd02 kubelet[24658]: W0717 17:38:28.223619   24658 docker_service.go:561] Hairpin mode set to "promiscuous-bri
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.223633   24658 docker_service.go:238] Hairpin mode set to "hairpin-veth"
7月 17 17:38:28 bd02 kubelet[24658]: W0717 17:38:28.223743   24658 cni.go:213] Unable to update cni config: No networks found
7月 17 17:38:28 bd02 kubelet[24658]: E0717 17:38:28.225053   24658 reflector.go:126] k8s.io/kubernetes/pkg/kubelet/kubelet.go:
7月 17 17:38:28 bd02 kubelet[24658]: E0717 17:38:28.225160   24658 reflector.go:126] k8s.io/kubernetes/pkg/kubelet/kubelet.go:
7月 17 17:38:28 bd02 kubelet[24658]: E0717 17:38:28.225551   24658 reflector.go:126] k8s.io/kubernetes/pkg/kubelet/config/apis
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.225688   24658 docker_service.go:253] Docker cri networking managed by kub
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.229200   24658 docker_service.go:258] Docker Info: &{ID:CBLH:B63G:MQE2:L3A
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.229249   24658 docker_service.go:271] Setting cgroupDriver to cgroupfs
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.229298   24658 kubelet.go:632] Starting the GRPC server for the docker CRI
7月 17 17:38:28 bd02 kubelet[24658]: I0717 17:38:28.229316   24658 docker_server.go:59] Start dockershim grpc server
7月 17 17:38:28 bd02 kubelet[24658]: F0717 17:38:28.229437   24658 docker_service.go:412] Streaming server stopped unexpectedl
7月 17 17:38:28 bd02 systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a
7月 17 17:38:28 bd02 systemd[1]: Unit kubelet.service entered failed state.
7月 17 17:38:28 bd02 systemd[1]: kubelet.service failed.
7月 17 17:38:33 bd02 systemd[1]: kubelet.service holdoff time over, scheduling restart.

opsnull / follow-me-install-kubernetes-cluster

kubelet貌似一直在重启，百思不得其解啊 #465