shiftrepo / OperationBreakingDawn

The Unlicense
0 stars 0 forks source link

kubernetes&DockerOnVM #23

Closed MITLabo closed 5 years ago

MITLabo commented 5 years ago

openstackのベース層ではなく、本番ベアメタル適用を考慮したansibleベースでの環境作成用資産を構築する。

MITLabo commented 5 years ago

本家 https://kubernetes.io/docs/setup/independent/install-kubeadm/ https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

  1. masterノードを作成する。
  2. masterノードよりslaveノードのtokenを発行し、joinさせる。
MITLabo commented 5 years ago

だいぶAnsibleも慣れてきた。kubernetes完成。

NAMESPACE     NAME                                          READY   STATUS    RESTARTS   AGE     IP          NODE                  NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-d7t2x                      1/1     Running   0          2m20s   10.88.0.2   k8s000.master.shift   <none>           <none>
kube-system   coredns-86c58d9df4-g9rx4                      1/1     Running   0          2m20s   10.88.0.3   k8s000.master.shift   <none>           <none>
kube-system   etcd-k8s000.master.shift                      1/1     Running   0          95s     10.0.2.6    k8s000.master.shift   <none>           <none>
kube-system   kube-apiserver-k8s000.master.shift            1/1     Running   0          101s    10.0.2.6    k8s000.master.shift   <none>           <none>
kube-system   kube-controller-manager-k8s000.master.shift   1/1     Running   0          102s    10.0.2.6    k8s000.master.shift   <none>           <none>
kube-system   kube-flannel-ds-amd64-sljtr                   1/1     Running   0          99s     10.0.2.6    k8s000.master.shift   <none>           <none>
kube-system   kube-proxy-g65w9                              1/1     Running   0          2m20s   10.0.2.6    k8s000.master.shift   <none>           <none>
kube-system   kube-scheduler-k8s000.master.shift            1/1     Running   0          102s    10.0.2.6    k8s000.master.shift   <none>           <none>

こいつが厄介だな。 Swap disabled. You MUST disable swap in order for the kubelet to work properly. どうしようかな。サービス起動時の制限か。fstabいじっちゃうか

MITLabo commented 5 years ago

internalのIPで設定できるようにしたいので、利用する。

(Optional) Unless otherwise specified, kubeadm uses the network interface associated with the default gateway to advertise the master’s IP. To use a different network interface, specify the --apiserver-advertise-address= argument to kubeadm init. To deploy an IPv6 Kubernetes cluster using IPv6 addressing, you must specify an IPv6 address, for example

こうするにたどり着くのに時間を要した。

/etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS=--node-ip=172.16.107.10

workerをjoinしたが、IPが外部向けだ。でもworkerの参加はコマンド一発。すごい。
ただし、トークン発行の動的項目が含まれるコマンドになってしまうので、ansible泣かせだな。evalか。
とりあえず今は手動として一旦クローズとする。flannelのamd64が気になるが。。。

[root@node000 KD_on_VM]# kubectl get pods --all-namespaces -o wide
NAMESPACE     NAME                                          READY   STATUS    RESTARTS   AGE     IP              NODE                  NOMINATED NODE   READINESS GATES
kube-system   coredns-86c58d9df4-2gv6w                      1/1     Running   0          8m55s   10.88.0.3       k8s000.master.shift   <none>           <none>
kube-system   coredns-86c58d9df4-w2999                      1/1     Running   0          8m55s   10.88.0.2       k8s000.master.shift   <none>           <none>
kube-system   etcd-k8s000.master.shift                      1/1     Running   0          9m8s    172.16.107.10   k8s000.master.shift   <none>           <none>
kube-system   kube-apiserver-k8s000.master.shift            1/1     Running   0          9m8s    172.16.107.10   k8s000.master.shift   <none>           <none>
kube-system   kube-controller-manager-k8s000.master.shift   1/1     Running   0          9m8s    172.16.107.10   k8s000.master.shift   <none>           <none>
kube-system   kube-flannel-ds-amd64-2pqhw                   1/1     Running   0          8m55s   172.16.107.10   k8s000.master.shift   <none>           <none>
kube-system   kube-flannel-ds-amd64-pt2lv                   1/1     Running   0          32s     10.0.2.8        k8s001.slave.shift    <none>           <none>
kube-system   kube-proxy-ftvn7                              1/1     Running   0          32s     10.0.2.8        k8s001.slave.shift    <none>           <none>
kube-system   kube-proxy-njlc8                              1/1     Running   0          8m55s   172.16.107.10   k8s000.master.shift   <none>           <none>
kube-system   kube-scheduler-k8s000.master.shift            1/1     Running   0          9m8s    172.16.107.10   k8s000.master.shift   <none>           <none>
MITLabo commented 5 years ago

いやーやっとテストデプロイにたどり着いた。仮想はネットワーク力を問われるな。頭がウニになる。

[root@node000 roles]# kubectl get all
NAME                               READY   STATUS    RESTARTS   AGE
pod/hello-world-696b6b59bd-7g7wn   1/1     Running   1          68m
pod/hello-world-696b6b59bd-zlcxw   1/1     Running   1          68m
pod/nginx-5cd9c7f879-lfrmf         1/1     Running   1          83m

NAME                      TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
service/example-service   NodePort       10.109.217.12   <none>        8080:32507/TCP   66m
service/kubernetes        ClusterIP      10.96.0.1       <none>        443/TCP          124m
service/nginx             LoadBalancer   10.111.42.228   <pending>     80:31751/TCP     82m

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/hello-world   2/2     2            2           68m
deployment.apps/nginx         1/1     1            1           83m

NAME                                     DESIRED   CURRENT   READY   AGE
replicaset.apps/hello-world-696b6b59bd   2         2         2       68m
replicaset.apps/nginx-5cd9c7f879         1         1         1       83m
[root@node000 roles]# kubectl describe pod/nginx-5cd9c7f879-lfrmf
Name:               nginx-5cd9c7f879-lfrmf
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               k8s001.slave.shift/10.0.2.8
Start Time:         Mon, 14 Jan 2019 16:49:30 +0900
Labels:             app=nginx
                    pod-template-hash=5cd9c7f879
Annotations:        <none>
Status:             Running
IP:                 10.244.1.5
Controlled By:      ReplicaSet/nginx-5cd9c7f879
Containers:
  nginx:
    Container ID:   docker://0fa881171bafa8e8d9db23a030219f32026b74ddbf73975d0b69df42136682b3
    Image:          nginx:1
    Image ID:       docker-pullable://nginx@sha256:b543f6d0983fbc25b9874e22f4fe257a567111da96fd1d8f1b44315f1236398c
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Mon, 14 Jan 2019 18:05:14 +0900
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 14 Jan 2019 16:49:49 +0900
      Finished:     Mon, 14 Jan 2019 18:05:01 +0900
    Ready:          True
    Restart Count:  1
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-l2bps (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-l2bps:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-l2bps
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason          Age    From                         Message
  ----    ------          ----   ----                         -------
  Normal  SandboxChanged  8m16s  kubelet, k8s001.slave.shift  Pod sandbox changed, it will be killed and re-created.
  Normal  Pulled          8m15s  kubelet, k8s001.slave.shift  Container image "nginx:1" already present on machine
  Normal  Created         8m15s  kubelet, k8s001.slave.shift  Created container
  Normal  Started         8m15s  kubelet, k8s001.slave.shift  Started container
[root@node000 roles]# curl http://10.0.2.8:31751
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>