KubeKey offline upgrade failed to download helm binary KK离线升级Kubesphere版本，找不到Helm

zhouwenblog commented 2 years ago

What is version of KubeKey has the issue?

version.BuildInfo{Version:"1.2.0", GitCommit:"c3068bf", GitTreeState:"", GoVersion:"go1.16.9"}.

What is your os environment?

redhat 7.6

KubeKey config file

apiVersion: kubekey.kubesphere.io/v1alpha1
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: node1, address: 202.96.18.62, internalAddress: 202.96.18.62, user: root, password: Root_123}
  - {name: node2, address: 202.96.18.63, internalAddress: 202.96.18.63, user: root, password: Root_123}
  - {name: node3, address: 202.96.18.64, internalAddress: 202.96.18.64, user: root, password: Root_123}
  - {name: node4, address: 202.96.18.65, internalAddress: 202.96.18.65, user: root, password: Root_123}
  roleGroups:
    etcd:
    - node1
    master: 
    - node1
    worker:
    - node1
    - node2
    - node3
    - node4
  controlPlaneEndpoint:
    ##Internal loadbalancer for apiservers 
    #internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.21.5
    clusterName: cluster.local
  network:
    plugin: calico
    kubePodsCIDR: 10.233.64.0/18
    kubeServiceCIDR: 10.233.0.0/18
  registry:
    registryMirrors: []
    insecureRegistries: []
    privateRegistry: dockerhub.kubekey.local
  addons: []

---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
  name: ks-installer
  namespace: kubesphere-system
  labels:
    version: v3.2.0
spec:
  persistence:
    storageClass: ""
  authentication:
    jwtSecret: ""
  local_registry: ""
  # dev_tag: ""
  etcd:
    monitoring: false
    endpointIps: localhost
    port: 2379
    tlsEnable: true
  common:
    core:
      console:
        enableMultiLogin: true
        port: 30880
        type: NodePort
    # apiserver:
    #  resources: {}
    # controllerManager:
    #  resources: {}
    redis:
      enabled: false
      volumeSize: 2Gi
    openldap:
      enabled: false
      volumeSize: 2Gi
    minio:
      volumeSize: 20Gi
    monitoring:
      # type: external
      endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
      GPUMonitoring:
        enabled: false
    gpu:
      kinds:         
      - resourceName: "nvidia.com/gpu"
        resourceType: "GPU"
        default: true
    es:
      # master:
      #   volumeSize: 4Gi
      #   replicas: 1
      #   resources: {}
      # data:
      #   volumeSize: 20Gi
      #   replicas: 1
      #   resources: {}
      logMaxAge: 7
      elkPrefix: logstash
      basicAuth:
        enabled: false
        username: ""
        password: ""
      externalElasticsearchUrl: ""
      externalElasticsearchPort: ""
  alerting:
    enabled: false
    # thanosruler:
    #   replicas: 1
    #   resources: {}
  auditing:
    enabled: false
    # operator:
    #   resources: {}
    # webhook:
    #   resources: {}
  devops:
    enabled: false
    jenkinsMemoryLim: 2Gi
    jenkinsMemoryReq: 1500Mi
    jenkinsVolumeSize: 8Gi
    jenkinsJavaOpts_Xms: 512m
    jenkinsJavaOpts_Xmx: 512m
    jenkinsJavaOpts_MaxRAM: 2g
  events:
    enabled: false
    # operator:
    #   resources: {}
    # exporter:
    #   resources: {}
    # ruler:
    #   enabled: true
    #   replicas: 2
    #   resources: {}
  logging:
    enabled: false
    containerruntime: docker
    logsidecar:
      enabled: true
      replicas: 2
      # resources: {}
  metrics_server:
    enabled: false
  monitoring:
    storageClass: ""
    # kube_rbac_proxy:
    #   resources: {}
    # kube_state_metrics:
    #   resources: {}
    # prometheus:
    #   replicas: 1
    #   volumeSize: 20Gi
    #   resources: {}
    #   operator:
    #     resources: {}
    #   adapter:
    #     resources: {}
    # node_exporter:
    #   resources: {}
    # alertmanager:
    #   replicas: 1
    #   resources: {}
    # notification_manager:
    #   resources: {}
    #   operator:
    #     resources: {}
    #   proxy:
    #     resources: {}
    gpu:
      nvidia_dcgm_exporter:
        enabled: false
        # resources: {}
  multicluster:
    clusterRole: none 
  network:
    networkpolicy:
      enabled: false
    ippool:
      type: none
    topology:
      type: none
  openpitrix:
    store:
      enabled: false
  servicemesh:
    enabled: false
  kubeedge:
    enabled: false   
    cloudCore:
      nodeSelector: {"node-role.kubernetes.io/worker": ""}
      tolerations: []
      cloudhubPort: "10000"
      cloudhubQuicPort: "10001"
      cloudhubHttpsPort: "10002"
      cloudstreamPort: "10003"
      tunnelPort: "10004"
      cloudHub:
        advertiseAddress:
          - ""
        nodeLimit: "100"
      service:
        cloudhubNodePort: "30000"
        cloudhubQuicNodePort: "30001"
        cloudhubHttpsNodePort: "30002"
        cloudstreamNodePort: "30003"
        tunnelNodePort: "30004"
    edgeWatcher:
      nodeSelector: {"node-role.kubernetes.io/worker": ""}
      tolerations: []
      edgeWatcherAgent:
        nodeSelector: {"node-role.kubernetes.io/worker": ""}
        tolerations: []

A clear and concise description of what happend.

/kk upgrade -f config-sample.yaml

Relevant log output

INFO[16:00:51 CST] Upgrading kube cluster                       
INFO[16:00:51 CST] Start Upgrade: v1.20.6 -> v1.21.5            
INFO[16:00:51 CST] Downloading Installation Files               
INFO[16:00:51 CST] Downloading kubeadm ...                      
INFO[16:00:53 CST] Downloading kubelet ...                      
INFO[16:00:56 CST] Downloading kubectl ...                      
INFO[16:00:57 CST] Downloading helm ...                         
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: kubernetes-helm.pek3b.qingstor.com; 未知的错误

WARN[16:00:58 CST] Task failed ...                              
WARN[16:00:58 CST] error: Failed to download helm binary: curl -L -o /soft/kubesphere/v3.2.0/kubekey/v1.21.5/amd64/helm https://kubernetes-helm.pek3b.qingstor.com/linux-amd64/v3.6.3/helm 
Error: Failed to upgrade kube cluster: Failed to download helm binary: curl -L -o /soft/kubesphere/v3.2.0/kubekey/v1.21.5/amd64/helm https://kubernetes-helm.pek3b.qingstor.com/linux-amd64/v3.6.3/helm

Additional information

离线升级版本提前在外网的机器上下载好了KubeKey里面的离线文件kubekey/v1.21.5/amd64/ cni-plugins-linux-amd64-v0.8.6.tgz helm kubectl kubelet kubeadm 之前从3.1.0升级到3.1.1 离线操作没问题，能读到拷贝进去的文件，但是从3.1.1升级到3.2.0提示helm无法使用离线文件，内网环境没法连接互联网

24sama commented 2 years ago

The default helm version in Kubekey v1.2.0 is v3.6.3. Is the version different from your offline helm file?

pixiake commented 2 years ago

You can download those binary files by executingkk create cluster on a machine that can connect to the Internet. This is the easiest way.

BTW，KubeKey will support custom-built offline installation packages in the next major version.

kubesphere / kubekey