kubesphere / kubekey

Install Kubernetes/K3s only, both Kubernetes/K3s and KubeSphere, and related cloud-native add-ons, it supports all-in-one, multi-node, and HA 🔥 ⎈ 🐳
https://kubesphere.io
Apache License 2.0
2.28k stars 538 forks source link

arm64 无法安装 minio 镜像 #1749

Open shuqingzai opened 1 year ago

shuqingzai commented 1 year ago

What is version of KubeKey has the issue?

kk version: &version.Info{Major:"3", Minor:"0", GitVersion:"v3.0.7-dirty", GitCommit:"e755baf67198d565689d7207378174f429b508ba", GitTreeState:"dirty", BuildDate:"2023-01-18T01:59:52Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/arm64"}

What is your os environment?

ubuntu22.04 arm64

KubeKey config file

apiVersion: kubekey.kubesphere.io/v1alpha2
kind: Cluster
metadata:
  name: sample
spec:
  hosts:
  - {name: k8s01, address: 10.211.55.101, internalAddress: 10.211.55.101, user: test, password: "123456", arch: arm64}
  - {name: k8s02, address: 10.211.55.102, internalAddress: 10.211.55.102, user: test, password: "123456", arch: arm64}
  - {name: k8s03, address: 10.211.55.103, internalAddress: 10.211.55.103, user: test, password: "123456", arch: arm64}
  roleGroups:
    etcd:
    - k8s01
    control-plane:
    - k8s01
    worker:
    - k8s02
    - k8s03
  controlPlaneEndpoint:
    ## Internal loadbalancer for apiservers
    # internalLoadbalancer: haproxy

    domain: lb.kubesphere.local
    address: ""
    port: 6443
  kubernetes:
    version: v1.24.9
    clusterName: cluster.local
    autoRenewCerts: true
    containerManager: containerd
    proxyMode: ipvs
  etcd:
    type: kubekey
  network:
    plugin: calico
    kubePodsCIDR: 192.168.64.0/18
    kubeServiceCIDR: 192.168.0.0/18
    ## multus support. https://github.com/k8snetworkplumbingwg/multus-cni
    multusCNI:
      enabled: false
  registry:
    privateRegistry: ""
    namespaceOverride: ""
    registryMirrors: []
    insecureRegistries: []
  addons:
  - name: nfs-client
    namespace: kube-system
    sources:
      chart:
        name: nfs-client-provisioner
        repo: https://charts.kubesphere.io/main
        valuesFile: /k8s/cluster-config/custom-nfs-client-values.yaml  # or https://raw.githubusercontent.com/kubesphere/helm-charts/master/src/main/nfs-client-provisioner/values.yaml
        # values also supports parameter lists
        values:
          - storageClass.defaultClass=true
          - nfs.server=10.211.55.101
          - nfs.path=/nfs/k8s

---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:
  name: ks-installer
  namespace: kubesphere-system
  labels:
    version: v3.3.2
spec:
  persistence:
    storageClass: ""
  authentication:
    jwtSecret: ""
  zone: ""
  local_registry: ""
  namespace_override: ""
  # dev_tag: ""
  etcd:
    monitoring: true
    endpointIps: 10.211.55.101
    port: 2379
    tlsEnable: true
  common:
    core:
      console:
        enableMultiLogin: true
        port: 30880
        type: NodePort
    # apiserver:
    #  resources: {}
    # controllerManager:
    #  resources: {}
    redis:
      enabled: true
      volumeSize: 2Gi
    openldap:
      enabled: false
      volumeSize: 2Gi
    minio:
      volumeSize: 20Gi
    monitoring:
      # type: external
      endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090
      GPUMonitoring:
        enabled: false
    gpu:
      kinds:
      - resourceName: "nvidia.com/gpu"
        resourceType: "GPU"
        default: true
    es:
      # master:
      #   volumeSize: 4Gi
      #   replicas: 1
      #   resources: {}
      # data:
      #   volumeSize: 20Gi
      #   replicas: 1
      #   resources: {}
      logMaxAge: 7
      elkPrefix: logstash
      basicAuth:
        enabled: false
        username: ""
        password: ""
      externalElasticsearchHost: ""
      externalElasticsearchPort: ""
  alerting:
    enabled: true
    # thanosruler:
    #   replicas: 1
    #   resources: {}
  auditing:
    enabled: true
    # operator:
    #   resources: {}
    # webhook:
    #   resources: {}
  devops:
    enabled: true
    # resources: {}
    jenkinsMemoryLim: 8Gi
    jenkinsMemoryReq: 4Gi
    jenkinsVolumeSize: 8Gi
  events:
    enabled: false
    # operator:
    #   resources: {}
    # exporter:
    #   resources: {}
    # ruler:
    #   enabled: true
    #   replicas: 2
    #   resources: {}
  logging:
    enabled: false
    logsidecar:
      enabled: true
      replicas: 2
      # resources: {}
  metrics_server:
    enabled: false
  monitoring:
    storageClass: ""
    node_exporter:
      port: 9100
      # resources: {}
    # kube_rbac_proxy:
    #   resources: {}
    # kube_state_metrics:
    #   resources: {}
    # prometheus:
    #   replicas: 1
    #   volumeSize: 20Gi
    #   resources: {}
    #   operator:
    #     resources: {}
    # alertmanager:
    #   replicas: 1
    #   resources: {}
    # notification_manager:
    #   resources: {}
    #   operator:
    #     resources: {}
    #   proxy:
    #     resources: {}
    gpu:
      nvidia_dcgm_exporter:
        enabled: false
        # resources: {}
  multicluster:
    clusterRole: none
  network:
    networkpolicy:
      enabled: true
    ippool:
      type: calico
    topology:
      type: none
  openpitrix:
    store:
      enabled: true
  servicemesh:
    enabled: false
    istio:
      components:
        ingressGateways:
        - name: istio-ingressgateway
          enabled: false
        cni:
          enabled: false
  edgeruntime:
    enabled: false
    kubeedge:
      enabled: false
      cloudCore:
        cloudHub:
          advertiseAddress:
            - ""
        service:
          cloudhubNodePort: "30000"
          cloudhubQuicNodePort: "30001"
          cloudhubHttpsNodePort: "30002"
          cloudstreamNodePort: "30003"
          tunnelNodePort: "30004"
        # resources: {}
        # hostNetWork: false
      iptables-manager:
        enabled: true
        mode: "external"
        # resources: {}
      # edgeService:
      #   resources: {}
  terminal:
    timeout: 1000

A clear and concise description of what happend.

镜像不支持 arm64 ,容器无法启动

                         node.kubernetes.io/unreachable:NoExecute op=Exists for 300s

Events: Type Reason Age From Message


Warning FailedScheduling 3m32s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling. Normal Scheduled 3m30s default-scheduler Successfully assigned kubesphere-system/minio-746f646bfb-22nvf to k8s03 Normal Pulling 3m29s kubelet Pulling image "registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z" Normal Pulled 3m25s kubelet Successfully pulled image "registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z" in 4.460970778s Normal Created 2m34s (x4 over 3m25s) kubelet Created container minio Normal Started 2m34s (x4 over 3m25s) kubelet Started container minio Normal Pulled 2m34s (x3 over 3m24s) kubelet Container image "registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z" already present on machine Warning BackOff 2m2s (x12 over 3m23s) kubelet Back-off restarting failed container

Relevant log output

deployment.apps/ks-installer created
18:33:21 CST success: [k8s01]
18:33:21 CST [DeployKubeSphereModule] Add config to ks-installer manifests
18:33:21 CST success: [k8s01]
18:33:21 CST [DeployKubeSphereModule] Create the kubesphere namespace
18:33:21 CST success: [k8s01]
18:33:21 CST [DeployKubeSphereModule] Setup ks-installer config
18:33:21 CST stdout: [k8s01]
secret/kube-etcd-client-certs created
18:33:21 CST success: [k8s01]
18:33:21 CST [DeployKubeSphereModule] Apply ks-installer
18:33:23 CST stdout: [k8s01]
namespace/kubesphere-system unchanged
serviceaccount/ks-installer unchanged
customresourcedefinition.apiextensions.k8s.io/clusterconfigurations.installer.kubesphere.io unchanged
clusterrole.rbac.authorization.k8s.io/ks-installer unchanged
clusterrolebinding.rbac.authorization.k8s.io/ks-installer unchanged
deployment.apps/ks-installer unchanged
clusterconfiguration.installer.kubesphere.io/ks-installer created
18:33:23 CST success: [k8s01]
Please wait for the installation to complete:  >>--->

Additional information

image image

镜像: registry.cn-beijing.aliyuncs.com/kubesphereio/minio:RELEASE.2019-08-07T01-59-21Z

shuqingzai commented 1 year ago

kubesphere/ks-installer#2040