kubesphere / kubekey

Install Kubernetes/K3s only, both Kubernetes/K3s and KubeSphere, and related cloud-native add-ons, it supports all-in-one, multi-node, and HA 🔥 ⎈ 🐳
https://kubesphere.io
Apache License 2.0
2.37k stars 552 forks source link

离线部署k8s,使用自己部署的harbor(配置了https),无法部署成功 #1847

Open FumingD opened 1 year ago

FumingD commented 1 year ago

What is version of KubeKey has the issue?

3.0.7

What is your os environment?

centos 7.9

KubeKey config file

apiVersion: kubekey.kubesphere.io/v1alpha2

kind: Cluster

metadata:

  name: sample

spec:

  hosts:

  - {name: master1, address: 192.168.0.3, internalAddress: 192.168.0.3, user: root, password: "<REPLACE_WITH_YOUR_ACTUAL_PASSWORD>"}
  - {name: master2, address: 192.168.0.4, internalAddress: 192.168.0.4, user: root, password: "<REPLACE_WITH_YOUR_ACTUAL_PASSWORD>"}
  - {name: master2, address: 192.168.0.6, internalAddress: 192.168.0.6, user: root, password: "<REPLACE_WITH_YOUR_ACTUAL_PASSWORD>"}

  - {name: node1, address: 192.168.0.5, internalAddress: 192.168.0.5, user: root, password: "<REPLACE_WITH_YOUR_ACTUAL_PASSWORD>"}

  roleGroups:

    etcd:

    - master1
    - master2
    - master3
    control-plane:
    - master1
    - master2
    - master3

    worker:

    - node1

    # 如需使用 kk 自动部署镜像仓库,请设置该主机组 (建议仓库与集群分离部署,减少相互影响)

    registry:

    - node1

  controlPlaneEndpoint:

    ## Internal loadbalancer for apiservers

    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local

    address: ""

    port: 6443

  kubernetes:

    version: v1.22.12

    clusterName: cluster.local

  network:

    plugin: calico

    kubePodsCIDR: 10.233.64.0/18

    kubeServiceCIDR: 10.233.0.0/18

    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni

    multusCNI:

      enabled: false

  registry:

    # 如需使用 kk 部署 harbor, 可将该参数设置为 harbor,不设置该参数且需使用 kk 创建容器镜像仓库,将默认使用docker registry。

    type: harbor

    # 如使用 kk 部署的 harbor 或其他需要登录的仓库,可设置对应仓库的auths,如使用 kk 创建的 docker registry 仓库,则无需配置该参数。

    # 注意:如使用 kk 部署 harbor,该参数请于 harbor 启动后设置。

    auths:

     "dockerhub.kubekey.local":

       username: admin

       password: Harbor12345

    # 设置集群部署时使用的私有仓库

    privateRegistry: "dockerhub.kubekey.local"

    namespaceOverride: "kubesphereio"

    registryMirrors: []

    insecureRegistries: []

  addons: []

A clear and concise description of what happend.

这个bug是在这个bug(https://github.com/kubesphere/kubekey/issues/1846),后续将harbor配置了https后出现的。 (好折腾啊,kubesphere的开发同事们,能一起看看我提出的问题不?这两天搞不定,我恐怕不得不尝试kubeadm 或者kubepray来部署了),说下我的操作步骤:

  1. harbor 参考官网配置https(https://goharbor.io/docs/2.0.0/install-config/configure-https/), 并做自测,使用浏览器访问https://192.168.0.9 (假设harbor服务器 ip是这个),可以访问,在集群的每个master和worker节点配置证书,/etc/docker/daemon.json 改为insecure-registries: ["https://dockerhub.kubekey.local", "https://192.168.0.9"], daemon.json 也不知道是否需要这样改,docker login验证,可以login。
  2. 执行 ./kk create cluster -f config-sample.yaml -a kubesphere.tar.gz --with-packages,命令。开始安装,但后续报如下的错误: CopyImageToRegistry ... pinging container response “https://dockerhub.kubekey.local/v2”, x509 certificate signed by unknown authority

Relevant log output

CopyImageToRegistry ... pinging container response “https://dockerhub.kubekey.local/v2”,  x509 certificate signed by unknown authority

因为离线环境日志不能拍照,不能拷贝,手抄了部分日志。

Additional information

No response

redscholar commented 1 year ago

It seems that your server does not trust the x509 certificate.

If the issue is caused by a self-signed certificate, you can try the following steps on CentOS 7:

  1. Copy the certificate file to the system:

    $ sudo cp /path/to/certfile /etc/pki/ca-trust/source/anchors/

  2. Update the certificate trust list:

    $ sudo update-ca-trust

    or

    $ sudo update-ca-trust force-enable

    Note: If force-enable does not solve the problem, try disabling all old certificates and CA certificates to disabled state:

    $ sudo update-ca-trust disable

    and then repeat the following command:

    $ sudo update-ca-trust force-enable

  3. Restart the server:

    $ sudo reboot

The self-signed certificate will be added to the trust list of the operating system, and you can now use it normally on CentOS 7.