wnxn commented 5 years ago

What would you like to be added: 各组件版本 OS Ubuntu 18.04.1 LTS 64 bit(bionic1x64c) [10/18 更新，OS 版本应与 QKE v1.1.0 一致] Kubernetes v1.15.4 Docker v18.06.2-ce KubeSphere v2.1.0 QingCloud CSI v1.1.0 Calico v3.8.2 Flannel v0.11.0 Etcd 3.2.24 Cloud Controller Manager v1.4.0 Coredns 1.3.1

Why is this needed:

wnxn commented 5 years ago

Kubeadm v1.15.5 config

https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2

Supported Types

apiVersion: kubeadm.k8s.io/v1beta2
kind: InitConfiguration

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration

apiVersion: kubeadm.k8s.io/v1beta2
kind: JoinConfiguration

init defaults

# kubeadm config print init-defaults
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 1.2.3.4
  bindPort: 6443
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: i-sds3ltd3
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.15.0
networking:
  dnsDomain: cluster.local
  serviceSubnet: 10.96.0.0/12
scheduler: {}

join defaults

# kubeadm config print join-defaults
apiVersion: kubeadm.k8s.io/v1beta2
caCertPath: /etc/kubernetes/pki/ca.crt
discovery:
  bootstrapToken:
    apiServerEndpoint: kube-apiserver:6443
    token: abcdef.0123456789abcdef
    unsafeSkipCAVerification: true
  timeout: 5m0s
  tlsBootstrapToken: abcdef.0123456789abcdef
kind: JoinConfiguration
nodeRegistration:
  criSocket: /var/run/dockershim.sock
  name: i-sds3ltd3
  taints: null

wnxn commented 5 years ago

Kubernetes v1.15.5 + Calico

# docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
calico/node                          v3.8.4              83b416d24205        3 days ago          191MB
calico/pod2daemon-flexvol            v3.8.4              207f157c99ac        3 days ago          9.37MB
calico/cni                           v3.8.4              20d7eefd5ce2        3 days ago          157MB
calico/kube-controllers              v3.8.4              079a722d508a        3 days ago          48.9MB
k8s.gcr.io/kube-apiserver            v1.15.5             e534b1952a0d        5 days ago          207MB
k8s.gcr.io/kube-proxy                v1.15.5             cbd7f21fec99        5 days ago          82.4MB
k8s.gcr.io/kube-controller-manager   v1.15.5             1399a72fa1a9        5 days ago          159MB
k8s.gcr.io/kube-scheduler            v1.15.5             fab2dded59dd        5 days ago          81.1MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180        9 months ago        40.3MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4f        10 months ago       258MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        22 months ago       742kB

wnxn commented 5 years ago

Kubernetes v1.15.5 + Flannel

# docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy                v1.15.5             cbd7f21fec99        5 days ago          82.4MB
k8s.gcr.io/kube-apiserver            v1.15.5             e534b1952a0d        5 days ago          207MB
k8s.gcr.io/kube-controller-manager   v1.15.5             1399a72fa1a9        5 days ago          159MB
k8s.gcr.io/kube-scheduler            v1.15.5             fab2dded59dd        5 days ago          81.1MB
quay.io/coreos/flannel               v0.11.0-amd64       ff281650a721        8 months ago        52.6MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180        9 months ago        40.3MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4f        10 months ago       258MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        22 months ago       742kB

wnxn commented 5 years ago

制作 QKE KVM 镜像脚本中会升级 Linux 内核制作 QKE v1.1.0 KVM 映像时，执行 update-pkg.sh, update-storage-client.sh 后会升级内核

root@i-zjcqksev:/opt/kubernetes/image# uname -a
Linux i-zjcqksev 4.15.0-30-generic #32-Ubuntu SMP Thu Jul 26 17:42:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
root@i-zjcqksev:/opt/kubernetes/image# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"

In QKE 1.0.1

QKE cl-kza6hlnq v1.0.1
root@i-62hj8rpq:~# uname -a
Linux i-62hj8rpq 4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
root@i-62hj8rpq:~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"

magicsong commented 5 years ago

测试HOSTNIC和coredns的兼容性时老是发生错误，具体错误：

log: exiting because of error: log: cannot create log: open /tmp/coredns.coredns-5c98db65d4-64wm6.unknownuser.log.ERROR.20191025-023311.1: no such file or directory

参考issue: https://github.com/coredns/coredns/issues/2464

如上issue中，这个问题存在与1.3.1（包括）之前所有版本，建议升级至较高版本。我测试用的1.4.0是OK的镜像地址为coredns/coredns:v1.4.0

/cc @wnxn