search

kelseyhightower / kubernetes-the-hard-way

Bootstrap Kubernetes the hard way. No scripts.
Apache License 2.0
41.31k stars 14.13k forks source link

Update 09-bootstrapping-kubernetes-workers.md #591

Closed rainmanh closed 4 years ago

rainmanh commented 4 years ago

The previous configuration exposes the following issue:

Jul 23 12:41:43 worker-2 kubelet[29002]: #011For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Jul 23 12:41:43 worker-2 kubelet[29002]: E0723 12:41:43.068772   29002 remote_runtime.go:81] Version from runtime service failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService
Jul 23 12:41:43 worker-2 kubelet[29002]: E0723 12:41:43.068808   29002 kuberuntime_manager.go:197] Get runtime version failed: get remote runtime typed version failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService
Jul 23 12:41:43 worker-2 kubelet[29002]: F0723 12:41:43.068821   29002 server.go:274] failed to run Kubelet: failed to create kubelet: get remote runtime typed version failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService
Jul 23 12:41:43 worker-2 systemd[1]: kubelet.service: Main process exited, code=exited, status=255/n/a
Jul 23 12:41:43 worker-2 systemd[1]: kubelet.service: Failed with result 'exit-code'.

otherwise using the default config and restarting containerd and the kubelet service seems to come back clean.

Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.416373   29204 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc0009ec250, {CONNECTING <nil>}
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.416562   29204 balancer_conn_wrappers.go:78] pickfirstBalancer: HandleSubConnStateChange: 0xc0009ec250, {READY <nil>}
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.417181   29204 factory.go:137] Registering containerd factory
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.417475   29204 factory.go:54] Registering systemd factory
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.417741   29204 factory.go:101] Registering Raw factory
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.417965   29204 manager.go:1158] Started watching for new ooms in manager
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.418772   29204 manager.go:272] Starting recovery of all containers
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.424747   29204 manager.go:277] Recovery completed
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.470205   29204 cpu_manager.go:184] [cpumanager] starting with none policy
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.470459   29204 cpu_manager.go:185] [cpumanager] reconciling every 10s
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.470564   29204 state_mem.go:36] [cpumanager] initializing new in-memory state store
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.472940   29204 state_mem.go:88] [cpumanager] updated default cpuset: ""
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.473060   29204 state_mem.go:96] [cpumanager] updated cpuset assignments: "map[]"
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.473156   29204 state_checkpoint.go:136] [cpumanager] state checkpoint: restored state from checkpoint
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.473241   29204 state_checkpoint.go:137] [cpumanager] state checkpoint: defaultCPUSet:
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.473331   29204 policy_none.go:43] [cpumanager] none policy: Start
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.474216   29204 manager.go:236] Starting Device Plugin manager
Jul 23 12:43:41 worker-2 kubelet[29204]: W0723 12:43:41.474348   29204 manager.go:597] Failed to retrieve checkpoint for "kubelet_internal_checkpoint": checkpoint is not found
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.474604   29204 manager.go:278] Serving device plugin registration server on "/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.474734   29204 plugin_watcher.go:54] Plugin Watcher Start at /var/lib/kubelet/plugins_registry
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.474907   29204 plugin_manager.go:112] The desired_state_of_world populator (plugin watcher) starts
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.474986   29204 plugin_manager.go:114] Starting Kubelet Plugin Manager
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.492781   29204 kubelet_node_status.go:294] Setting node annotation to enable volume controller attach/detach
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.494039   29204 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.494184   29204 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.494292   29204 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.494407   29204 kubelet_node_status.go:70] Attempting to register node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.508919   29204 kubelet.go:1908] SyncLoop (ADD, "api"): ""
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.509874   29204 kubelet_node_status.go:112] Node worker-2 was previously registered
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.510026   29204 kubelet_node_status.go:73] Successfully registered node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.512199   29204 kubelet_node_status.go:486] Recording NodeHasSufficientMemory event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.512408   29204 kubelet_node_status.go:486] Recording NodeHasNoDiskPressure event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.512512   29204 kubelet_node_status.go:486] Recording NodeHasSufficientPID event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.512609   29204 kubelet_node_status.go:486] Recording NodeReady event message for node worker-2
Jul 23 12:43:41 worker-2 kubelet[29204]: I0723 12:43:41.696619   29204 reconciler.go:157] Reconciler: start to sync state
Jul 23 12:48:41 worker-2 kubelet[29204]: I0723 12:48:41.415322   29204 kubelet.go:1316] Image garbage collection succeeded

This is:

No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:    18.04
Codename:   bionic

kubelet --version
Kubernetes v1.18.6
rainmanh commented 4 years ago

Grammatical typo made with the copy/paste of the code. Re-issuing in another commit.