Open ShigeRunner opened 5 years ago
クラスターなどを一旦削除した。
削除したサーバーの docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7cc3f496a45f mysql:5.6 "docker-entrypoint..." 5 months ago Up 8 weeks 0.0.0.0:3307->3306/tcp mysql_db_1
7698b45a9f00 httpd:2.4 "httpd-foreground" 5 months ago Up 8 weeks 0.0.0.0:443->443/tcp, 0.0.0.0:88->80/tcp mysql_web_1
ついでに、docker image も整理した。docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
development_php latest ed3f08c3f5d0 5 months ago 95.7MB
mysql 5.6 7edb93321b06 6 months ago 256MB
httpd 2.4 11426a19f1a2 6 months ago 178MB
nginx 1.15.1-alpine 920e5c5c8bed 7 months ago 18.6MB
php 7.2.7-fpm-alpine 9cf17fea14c0 7 months ago 78.3MB
rails latest 660f41442a27 2 years ago 840MB
mysql 5.7.14 4b3b6b994512 2 years ago 385MB
rancher/agent を再設定
docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
23dc73929869 rancher/k8s-dns-sidecar-amd64 "/sidecar --v=2 --..." About a minute ago Up About a minute k8s_sidecar_kube-dns-7588d5b5f5-krk89_kube-system_db85be7a-30d9-11e9-a464-989096991eb4_0
9c587087f983 rancher/k8s-dns-dnsmasq-nanny-amd64 "/dnsmasq-nanny -v..." About a minute ago Up About a minute k8s_dnsmasq_kube-dns-7588d5b5f5-krk89_kube-system_db85be7a-30d9-11e9-a464-989096991eb4_0
7933027d46f0 rancher/nginx-ingress-controller-defaultbackend "/server" About a minute ago Up About a minute k8s_default-http-backend_default-http-backend-797c5bc547-j7xbj_ingress-nginx_e4cf6870-30d9-11e9-a464-989096991eb4_0
4ba8b8ee5a08 rancher/metrics-server-amd64 "/metrics-server -..." About a minute ago Up About a minute k8s_metrics-server_metrics-server-97bc649d5-bnwkp_kube-system_df3feac3-30d9-11e9-a464-989096991eb4_0
36d9de8f3799 rancher/k8s-dns-kube-dns-amd64 "/kube-dns --domai..." About a minute ago Up About a minute k8s_kubedns_kube-dns-7588d5b5f5-krk89_kube-system_db85be7a-30d9-11e9-a464-989096991eb4_0
052d4c29422b rancher/cluster-proportional-autoscaler-amd64 "/cluster-proporti..." About a minute ago Up About a minute k8s_autoscaler_kube-dns-autoscaler-5db9bbb766-lq9q7_kube-system_dbffdc84-30d9-11e9-a464-989096991eb4_0
3c018509ad3a rancher/coreos-flannel "/opt/bin/flanneld..." About a minute ago Up About a minute k8s_kube-flannel_canal-t546r_kube-system_d87f3a99-30d9-11e9-a464-989096991eb4_0
e9845dc8154b f3d94aa8c942 "run.sh" 2 minutes ago Up 2 minutes k8s_cluster-register_cattle-cluster-agent-5c97c9f5bc-mhjdr_cattle-system_e9fa7ade-30d9-11e9-a464-989096991eb4_0
c6da734f0b98 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_cattle-cluster-agent-5c97c9f5bc-mhjdr_cattle-system_e9fa7ade-30d9-11e9-a464-989096991eb4_0
77eea89a6dd0 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_default-http-backend-797c5bc547-j7xbj_ingress-nginx_e4cf6870-30d9-11e9-a464-989096991eb4_0
479d4de5a203 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_metrics-server-97bc649d5-bnwkp_kube-system_df3feac3-30d9-11e9-a464-989096991eb4_0
d463818d73de rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_kube-dns-7588d5b5f5-krk89_kube-system_db85be7a-30d9-11e9-a464-989096991eb4_0
f944e207be95 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_kube-dns-autoscaler-5db9bbb766-lq9q7_kube-system_dbffdc84-30d9-11e9-a464-989096991eb4_0
150a7fd18a9c f3d94aa8c942 "run.sh" 2 minutes ago Up 2 minutes k8s_agent_cattle-node-agent-jrstp_cattle-system_ea0c18a3-30d9-11e9-a464-989096991eb4_0
33a7d1c81665 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_cattle-node-agent-jrstp_cattle-system_ea0c18a3-30d9-11e9-a464-989096991eb4_0
bd427b27c1cd rancher/calico-cni "/install-cni.sh" 2 minutes ago Up 2 minutes k8s_install-cni_canal-t546r_kube-system_d87f3a99-30d9-11e9-a464-989096991eb4_0
69f6a8f6db7c rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_nginx-ingress-controller-s8mxb_ingress-nginx_e4b15d4e-30d9-11e9-a464-989096991eb4_0
de23f75acad1 rancher/calico-node "start_runit" 2 minutes ago Up 2 minutes k8s_calico-node_canal-t546r_kube-system_d87f3a99-30d9-11e9-a464-989096991eb4_0
e5e1c6966ea7 rancher/pause-amd64:3.1 "/pause" 2 minutes ago Up 2 minutes k8s_POD_canal-t546r_kube-system_d87f3a99-30d9-11e9-a464-989096991eb4_0
1bbc9fc622b9 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/en..." 3 minutes ago Up 3 minutes kube-proxy
bc27bf245af3 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/en..." 3 minutes ago Up 3 minutes kubelet
b0c189574159 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/en..." 3 minutes ago Up 3 minutes kube-scheduler
687c84c08468 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/en..." 3 minutes ago Up 3 minutes kube-controller-manager
dc319804c095 rancher/hyperkube:v1.11.6-rancher1 "/opt/rke-tools/en..." 3 minutes ago Up 3 minutes kube-apiserver
defe1ca5ed8e rancher/rke-tools:v0.1.15 "/opt/rke-tools/rk..." 4 minutes ago Up 4 minutes etcd-rolling-snapshots
6f833bff9d02 rancher/coreos-etcd:v3.2.18 "/usr/local/bin/et..." 4 minutes ago Up 4 minutes etcd
7cc3f496a45f mysql:5.6 "docker-entrypoint..." 5 months ago Up 8 weeks 0.0.0.0:3307->3306/tcp mysql_db_1
7698b45a9f00 httpd:2.4 "httpd-foreground" 5 months ago Up 8 weeks 0.0.0.0:443->443/tcp, 0.0.0.0:88->80/tcp mysql_web_1
docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
rancher/rancher-agent v2.1.6 f3d94aa8c942 2 weeks ago 296MB
rancher/hyperkube v1.11.6-rancher1 ad8f7f061355 8 weeks ago 984MB
rancher/rke-tools v0.1.15 c822ad0496a4 4 months ago 123MB
development_php latest ed3f08c3f5d0 5 months ago 95.7MB
mysql 5.6 7edb93321b06 6 months ago 256MB
httpd 2.4 11426a19f1a2 6 months ago 178MB
rancher/nginx-ingress-controller 0.16.2-rancher1 df4469c42185 6 months ago 364MB
nginx 1.15.1-alpine 920e5c5c8bed 7 months ago 18.6MB
php 7.2.7-fpm-alpine 9cf17fea14c0 7 months ago 78.3MB
rancher/calico-node v3.1.3 7eca10056c8e 8 months ago 248MB
rancher/calico-cni v3.1.3 9f355e076ea7 8 months ago 68.8MB
rancher/k8s-dns-dnsmasq-nanny-amd64 1.14.10 6816817d9dce 10 months ago 40.4MB
rancher/k8s-dns-kube-dns-amd64 1.14.10 55ffe31ac578 10 months ago 49.5MB
rancher/k8s-dns-sidecar-amd64 1.14.10 8a7739f672b4 10 months ago 41.6MB
rancher/coreos-etcd v3.2.18 e21fb69683f3 10 months ago 37.2MB
rancher/coreos-flannel v0.10.0 f0fad859c909 12 months ago 44.6MB
rancher/pause-amd64 3.1 da86e6ba6ca1 14 months ago 742kB
rancher/metrics-server-amd64 v0.2.1 9801395070f3 14 months ago 42.5MB
rancher/nginx-ingress-controller-defaultbackend 1.4 846921f0fe0e 15 months ago 4.84MB
rails latest 660f41442a27 2 years ago 840MB
rancher/cluster-proportional-autoscaler-amd64 1.0.0 e183460c484d 2 years ago 48.2MB
mysql 5.7.14 4b3b6b994512 2 years ago 385MB
カタログ から longhorn を設定
Longhorn Kubernetes Driver を csi で設定 → エラー変わらず
Longhorn Kubernetes Driver を flexvolume で設定 → エラーなく、起動している感じ。
カタログから MySQL を設定 → 起動できた。
再度、カタログから longhorn を設定 条件:1. Longhorn Kubernetes Driver を csi で設定
longhorn-csi-plugin で、エラー発生 使用イメージ:rancher/longhorn-manager:v0.3.3
CrashLoopBackOff: Back-off 1m20s restarting failed container=driver-registrar pod=longhorn-csi-plugin-jtpwj_longhorn-system(e0f3ea4b-30e0-11e9-a464-989096991eb4)
kubectl version
を確認
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.5", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-21T11:46:00Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.6", GitCommit:"b1d75deca493a24a2f87eb1efde1a569e52fc8d9", GitTreeState:"clean", BuildDate:"2018-12-16T04:30:10Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
物理サーバーの状態を確認
cat /etc/os-release
を確認
NAME="Ubuntu"
VERSION="16.04.5 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.5 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial
uname -a
を確認
Linux ubuntu 4.4.0-140-generic #166-Ubuntu SMP Wed Nov 14 20:09:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
cat /proc/version
を確認
Linux version 4.4.0-140-generic (buildd@lcy01-amd64-006) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10) ) #166-Ubuntu SMP Wed Nov 14 20:09:47 UTC 2018
docker version
を確認
Client:
Version: 17.05.0-ce
API version: 1.29
Go version: go1.7.5
Git commit: 89658be
Built: Thu May 4 22:10:54 2017
OS/Arch: linux/amd64
Server: Version: 17.05.0-ce API version: 1.29 (minimum version 1.12) Go version: go1.7.5 Git commit: 89658be Built: Thu May 4 22:10:54 2017 OS/Arch: linux/amd64 Experimental: false
以下のエラーが表示される。
Containers with unready status: [driver-registrar longhorn-csi-plugin]
→ 未ステータスのコンテナ:[driver-registrar longhorn-csi-plugin]イメージ:rancher/longhorn-manager:v0.3.3 で以下のログが表示される。
以上がメインのエラーとなっていると思われる。
そのエラーに関連して、
CrashLoopBackOff: Back-off 5m0s restarting failed container=driver-registrar pod=longhorn-csi-plugin-t2h6l_longhorn-system(cd835da8-303d-11e9-a52d-989096991eb4)
Containers with unready status: [csi-attacher]
→ 未ステータスのコンテナ:[csi-attacher]が、起動しない・・・しても、落ちるを繰り返す。