search

canonical / multipass

Multipass orchestrates virtual Ubuntu instances
https://multipass.run
GNU General Public License v3.0
7.88k stars 651 forks source link

In Apple M1 architecture, install multupass and deploy k3s by k3d, but docker run rancher failed #2510

Closed gamelife1314 closed 2 years ago

gamelife1314 commented 2 years ago

I don't know where to go for help with this problem, and I don't know if it's related to multipass。

multipass version:

image

docker run rancher/rancher:v2.5.8-linux-arm64

docker run -d --restart=unless-stopped -p 58880:80 -p 58443:443 --privileged --cgroupns host rancher/rancher:v2.5.8-linux-arm64

meet the following error:

2022/04/05 10:22:48 [INFO] Rancher version v2.5.8 (cf16ca13d) is starting
2022/04/05 10:22:48 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:false AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLevel:0 Agent:false Features: ClusterRegistry:}
2022/04/05 10:22:48 [INFO] Listening on /tmp/log.sock
2022/04/05 10:22:48 [INFO] Running etcd --data-dir=management-state/etcd --heartbeat-interval=500 --election-timeout=5000
running etcd on unsupported architecture "arm64" since ETCD_UNSUPPORTED_ARCH is set
2022-04-05 10:22:48.185398 W | pkg/flags: unrecognized environment variable ETCD_URL_arm64=https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-arm64.tar.gz
2022-04-05 10:22:48.185413 W | pkg/flags: unrecognized environment variable ETCD_URL_amd64=https://github.com/etcd-io/etcd/releases/download/v3.4.3/etcd-v3.4.3-linux-amd64.tar.gz
2022-04-05 10:22:48.185416 W | pkg/flags: unrecognized environment variable ETCD_URL=ETCD_URL_arm64
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2022-04-05 10:22:48.185427 I | etcdmain: etcd Version: 3.4.3
2022-04-05 10:22:48.185430 I | etcdmain: Git SHA: 3cf2f69b5
2022-04-05 10:22:48.185431 I | etcdmain: Go Version: go1.12.12
2022-04-05 10:22:48.185433 I | etcdmain: Go OS/Arch: linux/arm64
2022-04-05 10:22:48.185435 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2022-04-05 10:22:48.185469 N | etcdmain: the server is already initialized as member before, starting as etcd member...
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap' flag instead
2022-04-05 10:22:48.185751 I | embed: name = default
2022-04-05 10:22:48.185757 I | embed: data dir = management-state/etcd
2022-04-05 10:22:48.185758 I | embed: member dir = management-state/etcd/member
2022-04-05 10:22:48.185760 I | embed: heartbeat = 500ms
2022-04-05 10:22:48.185761 I | embed: election = 5000ms
2022-04-05 10:22:48.185763 I | embed: snapshot count = 100000
2022-04-05 10:22:48.185766 I | embed: advertise client URLs = http://localhost:2379
2022-04-05 10:22:48.185769 I | embed: initial advertise peer URLs = http://localhost:2380
2022-04-05 10:22:48.185772 I | embed: initial cluster =
2022-04-05 10:22:48.186277 I | etcdserver: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 9
raft2022/04/05 10:22:48 INFO: 8e9e05c52164694d switched to configuration voters=()
raft2022/04/05 10:22:48 INFO: 8e9e05c52164694d became follower at term 3
raft2022/04/05 10:22:48 INFO: newRaft 8e9e05c52164694d [peers: [], term: 3, commit: 9, applied: 0, lastindex: 9, lastterm: 3]
2022-04-05 10:22:48.188284 W | auth: simple token is not cryptographically signed
2022-04-05 10:22:48.189081 I | etcdserver: starting server... [version: 3.4.3, cluster version: to_be_decided]
2022-04-05 10:22:48.190244 I | embed: listening for peers on 127.0.0.1:2380
raft2022/04/05 10:22:48 INFO: 8e9e05c52164694d switched to configuration voters=(10276657743932975437)
2022-04-05 10:22:48.190482 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2022-04-05 10:22:48.190538 N | etcdserver/membership: set the initial cluster version to 3.4
2022-04-05 10:22:48.190555 I | etcdserver/api: enabled capabilities for version 3.4
raft2022/04/05 10:22:57 INFO: 8e9e05c52164694d is starting a new election at term 3
raft2022/04/05 10:22:57 INFO: 8e9e05c52164694d became candidate at term 4
raft2022/04/05 10:22:57 INFO: 8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 4
raft2022/04/05 10:22:57 INFO: 8e9e05c52164694d became leader at term 4
raft2022/04/05 10:22:57 INFO: raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 4
2022-04-05 10:22:57.187130 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
2022-04-05 10:22:57.187558 I | embed: ready to serve client requests
2022-04-05 10:22:57.188135 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!
2022/04/05 10:22:57 [INFO] Waiting for server to become available: Get "https://127.0.0.1:6443/version?timeout=15m0s": dial tcp 127.0.0.1:6443: connect: connection refused
time="2022-04-05T10:22:57.316746281Z" level=info msg="Starting k3s v1.19.8+k3s1 (95fc76b2)"
time="2022-04-05T10:22:57.316893863Z" level=info msg="Cluster bootstrap already complete"
time="2022-04-05T10:22:57.324711535Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=unknown --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --enable-admission-plugins=NodeRestriction --etcd-servers=http://localhost:2379 --insecure-port=0 --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=k3s --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-cluster-ip-range=10.43.0.0/16 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
I0405 10:22:57.325514      27 server.go:652] external host was not specified, using 172.17.0.3
I0405 10:22:57.325830      27 server.go:177] Version: v1.19.8+k3s1
I0405 10:22:57.328503      27 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0405 10:22:57.328512      27 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0405 10:22:57.329172      27 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0405 10:22:57.329180      27 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0405 10:22:57.343018      27 master.go:271] Using reconciler: lease
W0405 10:22:57.536146      27 genericapiserver.go:412] Skipping API batch/v2alpha1 because it has no resources.
W0405 10:22:57.543623      27 genericapiserver.go:412] Skipping API discovery.k8s.io/v1alpha1 because it has no resources.
W0405 10:22:57.551851      27 genericapiserver.go:412] Skipping API node.k8s.io/v1alpha1 because it has no resources.
W0405 10:22:57.561237      27 genericapiserver.go:412] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
W0405 10:22:57.563210      27 genericapiserver.go:412] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
W0405 10:22:57.573503      27 genericapiserver.go:412] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
W0405 10:22:57.597976      27 genericapiserver.go:412] Skipping API apps/v1beta2 because it has no resources.
W0405 10:22:57.597995      27 genericapiserver.go:412] Skipping API apps/v1beta1 because it has no resources.
I0405 10:22:57.603737      27 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0405 10:22:57.603750      27 plugins.go:161] Loaded 10 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
time="2022-04-05T10:22:57.608074561Z" level=info msg="Running kube-scheduler --address=127.0.0.1 --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --port=10251 --profiling=false --secure-port=0"
I0405 10:22:57.608405      27 registry.go:173] Registering SelectorSpread plugin
I0405 10:22:57.608480      27 registry.go:173] Registering SelectorSpread plugin
time="2022-04-05T10:22:57.608888844Z" level=info msg="Running kube-controller-manager --address=127.0.0.1 --allocate-node-cidrs=true --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --cluster-signing-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --port=10252 --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=0 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.key --use-service-account-credentials=true"
time="2022-04-05T10:22:57.609746711Z" level=info msg="Node token is available at /var/lib/rancher/k3s/server/token"
time="2022-04-05T10:22:57.609762877Z" level=info msg="To join node to cluster: k3s agent -s https://172.17.0.3:6443 -t ${NODE_TOKEN}"
time="2022-04-05T10:22:57.610551453Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
time="2022-04-05T10:22:57.610570619Z" level=info msg="Run: k3s kubectl"
time="2022-04-05T10:22:57.610778909Z" level=warning msg="Failed to find cpuset cgroup, you may need to add \"cgroup_enable=cpuset\" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)"
time="2022-04-05T10:22:57.610790992Z" level=error msg="Failed to find memory cgroup, you may need to add \"cgroup_memory=1 cgroup_enable=memory\" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)"
time="2022-04-05T10:22:57.610798533Z" level=fatal msg="failed to find memory cgroup, you may need to add \"cgroup_memory=1 cgroup_enable=memory\" to your linux cmdline (/boot/cmdline.txt on a Raspberry Pi)"
2022/04/05 10:22:57 [FATAL] k3s exited with: exit status 1
Saviq commented 2 years ago

@gamelife1314 this suggested it required some kernel config:

Failed to find cpuset cgroup, you may need to add \"cgroup_enable=cpuset\" to your linux cmdline Failed to find memory cgroup, you may need to add \"cgroup_memory=1 cgroup_enable=memory\" to your linux cmdline

But I did that:

$ sudo tee /etc/default/grub.d/99-cgroup.cfg <<EOF
# Enable cgroups
GRUB_CMDLINE_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_DEFAULT} cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory"
EOF
$ sudo update-grub
$ sudo reboot
$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-5.13.0-35-generic root=UUID=b41c3be7-a045-4698-9527-ee37ee088f70 ro console=tty1 console=ttyS0 cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory

And it didn't help. You'll need to reach out to k3s to see what can help here. Multipass is out of the way here.

gamelife1314 commented 2 years ago

OK，谢谢