kubernetes / k8s.io

Code and configuration to manage Kubernetes project infrastructure, including various *.k8s.io sites
https://git.k8s.io/community/sig-k8s-infra
Apache License 2.0
735 stars 822 forks source link

Kubeadm v1.24 installation failed on Red Hat-based distributions (CentOS7) using the new package repositories hosted at pkgs.k8s.io #6061

Closed nitinjagjivan closed 1 year ago

nitinjagjivan commented 1 year ago

What keywords did you search in kubeadm issues before filing this one?

If you have found any duplicates, you should instead reply there and close this page.

If you have not found any duplicates, delete this section and continue on.

Is this a BUG REPORT or FEATURE REQUEST?

Choose one: BUG REPORT

Versions

kubeadm version (use kubeadm version): 1.24.0

Environment:

What happened?

I am following instructions mentioned at https://v1-24.docs.kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/. Kubeadm install fails with below error:

$ sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes Loaded plugins: fastestmirror Repository 'docker-ce-edge' is missing name in configuration, using id Loading mirror speeds from cached hostfile

Total size: 45 M Total download size: 38 M Installed size: 249 M Downloading packages: Delta RPMs disabled because /usr/bin/applydeltarpm not installed. kubectl-1.28.3-150500.1.1.x86_ FAILED https://pkgs.k8s.io/core%3A/stable%3A/v1.24/rpm/x86_64/kubectl-1.28.3-150500.1.1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden 0.0 B/s | 0 B --:--:-- ETA Trying other mirror. To address this issue please refer to the below wiki article

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

kubeadm-1.28.3-150500.1.1.x86_ FAILED https://pkgs.k8s.io/core%3A/stable%3A/v1.24/rpm/x86_64/kubeadm-1.28.3-150500.1.1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden 0.0 B/s | 0 B --:--:-- ETA Trying other mirror. kubelet-1.28.3-150500.1.1.x86_ FAILED https://pkgs.k8s.io/core%3A/stable%3A/v1.24/rpm/x86_64/kubelet-1.28.3-150500.1.1.x86_64.rpm: [Errno 14] HTTPS Error 403 - Forbidden 0.0 B/s | 0 B --:--:-- ETA Trying other mirror.

Error downloading packages: kubectl-1.28.3-150500.1.1.x86_64: [Errno 256] No more mirrors to try. kubelet-1.28.3-150500.1.1.x86_64: [Errno 256] No more mirrors to try. kubeadm-1.28.3-150500.1.1.x86_64: [Errno 256] No more mirrors to try.

What you expected to happen?

v1.24.0 is successfully installed

How to reproduce it (as minimally and precisely as possible)?

Just follow the instructions mentioned at https://v1-24.docs.kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Anything else we need to know?

It works well with v1.28.3

nitinjagjivan commented 1 year ago

$ cat /etc/yum.repos.d/kubernetes.repo

[kubernetes] name=Kubernetes baseurl=https://pkgs.k8s.io/core:/stable:/v1.24/rpm/ enabled=1 gpgcheck=1 gpgkey=https://pkgs.k8s.io/core:/stable:/v1.24/rpm/repodata/repomd.xml.key exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni

chendave commented 1 year ago

This doesn't seems like a problem of kubeadm itself.

neolit123 commented 1 year ago

@nitinjagjivan you have some sort of connectivity issue.

the 1.24 install works fine for me. check these logs:

``` $ docker run -it fedora:40 bash [root@04a525576548 /]# cat <" Fingerprint: DE15 B144 86CD 377B 9E87 6E1A 2346 54DA 9A29 6436 From : https://pkgs.k8s.io/core:/stable:/v1.24/rpm/repodata/repomd.xml.key Key imported successfully Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : libmnl-1.0.5-3.fc39.x86_64 1/41 Installing : libnfnetlink-1.0.1-24.fc39.x86_64 2/41 Installing : libnetfilter_conntrack-1.0.9-2.fc39.x86_64 3/41 Installing : libfdisk-2.39.2-1.fc40.x86_64 4/41 Installing : libbpf-2:1.2.0-1.fc40.x86_64 5/41 Installing : iptables-libs-1.8.10-2.fc40.x86_64 6/41 Installing : libnetfilter_queue-1.0.5-5.fc39.x86_64 7/41 Installing : ethtool-2:6.5-1.fc40.x86_64 8/41 Installing : libnetfilter_cthelper-1.0.0-24.fc39.x86_64 9/41 Installing : libnetfilter_cttimeout-1.0.0-22.fc39.x86_64 10/41 Installing : kubectl-1.24.17-150500.1.1.x86_64 11/41 Installing : cri-tools-1.26.0-150500.1.1.x86_64 12/41 Installing : xkeyboard-config-2.40-1.fc40.noarch 13/41 Installing : libxkbcommon-1.6.0-1.fc40.x86_64 14/41 Installing : socat-1.7.4.4-3.fc39.x86_64 15/41 Installing : qrencode-libs-4.1.1-5.fc39.x86_64 16/41 Installing : psmisc-23.6-4.fc39.x86_64 17/41 Installing : iproute-6.5.0-1.fc40.x86_64 18/41 Installing : libseccomp-2.5.3-6.fc39.x86_64 19/41 Installing : libargon2-20190702-3.fc39.x86_64 20/41 Installing : kmod-libs-30-6.fc39.x86_64 21/41 Installing : iptables-legacy-libs-1.8.10-2.fc40.x86_64 22/41 Installing : iptables-legacy-1.8.10-2.fc40.x86_64 23/41 Running scriptlet: iptables-legacy-1.8.10-2.fc40.x86_64 23/41 Installing : ebtables-legacy-2.0.11-14.fc39.x86_64 24/41 Running scriptlet: ebtables-legacy-2.0.11-14.fc39.x86_64 24/41 Installing : diffutils-3.10-3.fc39.x86_64 25/41 Installing : dbus-common-1:1.14.10-1.fc40.noarch 26/41 Running scriptlet: dbus-common-1:1.14.10-1.fc40.noarch 26/41 Running scriptlet: dbus-broker-33-2.fc39.x86_64 27/41 Installing : dbus-broker-33-2.fc39.x86_64 27/41 Running scriptlet: dbus-broker-33-2.fc39.x86_64 27/41 Installing : dbus-1:1.14.10-1.fc40.x86_64 28/41 Installing : device-mapper-libs-1.02.196-1.fc39.x86_64 29/41 Installing : cryptsetup-libs-2.6.1-3.fc39.x86_64 30/41 Installing : device-mapper-1.02.196-1.fc39.x86_64 31/41 Running scriptlet: libutempter-1.2.1-10.fc39.x86_64 32/41 Installing : libutempter-1.2.1-10.fc39.x86_64 32/41 Installing : systemd-networkd-254.5-2.fc40.x86_64 33/41 Running scriptlet: systemd-networkd-254.5-2.fc40.x86_64 33/41 Installing : systemd-pam-254.5-2.fc40.x86_64 34/41 Installing : systemd-resolved-254.5-2.fc40.x86_64 35/41 Running scriptlet: systemd-resolved-254.5-2.fc40.x86_64 35/41 Installing : util-linux-2.39.2-1.fc40.x86_64 36/41 warning: /etc/adjtime created as /etc/adjtime.rpmnew Installing : systemd-254.5-2.fc40.x86_64 37/41 Running scriptlet: systemd-254.5-2.fc40.x86_64 37/41 Creating group 'input' with GID 104. Creating group 'kvm' with GID 36. Creating group 'render' with GID 105. Creating group 'sgx' with GID 106. Creating group 'systemd-journal' with GID 190. Creating group 'systemd-network' with GID 192. Creating user 'systemd-network' (systemd Network Management) with UID 192 and GID 192. Creating group 'systemd-oom' with GID 999. Creating user 'systemd-oom' (systemd Userspace OOM Killer) with UID 999 and GID 999. Creating group 'systemd-resolve' with GID 193. Creating user 'systemd-resolve' (systemd Resolver) with UID 193 and GID 193. Installing : containernetworking-plugins-1.3.0-3.fc39.x86_64 38/41 Installing : conntrack-tools-1.4.7-3.fc39.x86_64 39/41 Running scriptlet: conntrack-tools-1.4.7-3.fc39.x86_64 39/41 Installing : kubelet-1.24.17-150500.1.1.x86_64 40/41 Running scriptlet: kubelet-1.24.17-150500.1.1.x86_64 40/41 System has not been booted with systemd as init system (PID 1). Can't operate. Failed to connect to bus: Host is down Installing : kubeadm-1.24.17-150500.1.1.x86_64 41/41 Running scriptlet: systemd-resolved-254.5-2.fc40.x86_64 41/41 Running scriptlet: kubeadm-1.24.17-150500.1.1.x86_64 41/41 Verifying : conntrack-tools-1.4.7-3.fc39.x86_64 1/41 Verifying : containernetworking-plugins-1.3.0-3.fc39.x86_64 2/41 Verifying : cryptsetup-libs-2.6.1-3.fc39.x86_64 3/41 Verifying : dbus-1:1.14.10-1.fc40.x86_64 4/41 Verifying : dbus-broker-33-2.fc39.x86_64 5/41 Verifying : dbus-common-1:1.14.10-1.fc40.noarch 6/41 Verifying : device-mapper-1.02.196-1.fc39.x86_64 7/41 Verifying : device-mapper-libs-1.02.196-1.fc39.x86_64 8/41 Verifying : diffutils-3.10-3.fc39.x86_64 9/41 Verifying : ebtables-legacy-2.0.11-14.fc39.x86_64 10/41 Verifying : ethtool-2:6.5-1.fc40.x86_64 11/41 Verifying : iproute-6.5.0-1.fc40.x86_64 12/41 Verifying : iptables-legacy-1.8.10-2.fc40.x86_64 13/41 Verifying : iptables-legacy-libs-1.8.10-2.fc40.x86_64 14/41 Verifying : iptables-libs-1.8.10-2.fc40.x86_64 15/41 Verifying : kmod-libs-30-6.fc39.x86_64 16/41 Verifying : libargon2-20190702-3.fc39.x86_64 17/41 Verifying : libbpf-2:1.2.0-1.fc40.x86_64 18/41 Verifying : libfdisk-2.39.2-1.fc40.x86_64 19/41 Verifying : libmnl-1.0.5-3.fc39.x86_64 20/41 Verifying : libnetfilter_conntrack-1.0.9-2.fc39.x86_64 21/41 Verifying : libnetfilter_cthelper-1.0.0-24.fc39.x86_64 22/41 Verifying : libnetfilter_cttimeout-1.0.0-22.fc39.x86_64 23/41 Verifying : libnetfilter_queue-1.0.5-5.fc39.x86_64 24/41 Verifying : libnfnetlink-1.0.1-24.fc39.x86_64 25/41 Verifying : libseccomp-2.5.3-6.fc39.x86_64 26/41 Verifying : libutempter-1.2.1-10.fc39.x86_64 27/41 Verifying : libxkbcommon-1.6.0-1.fc40.x86_64 28/41 Verifying : psmisc-23.6-4.fc39.x86_64 29/41 Verifying : qrencode-libs-4.1.1-5.fc39.x86_64 30/41 Verifying : socat-1.7.4.4-3.fc39.x86_64 31/41 Verifying : systemd-254.5-2.fc40.x86_64 32/41 Verifying : systemd-networkd-254.5-2.fc40.x86_64 33/41 Verifying : systemd-pam-254.5-2.fc40.x86_64 34/41 Verifying : systemd-resolved-254.5-2.fc40.x86_64 35/41 Verifying : util-linux-2.39.2-1.fc40.x86_64 36/41 Verifying : xkeyboard-config-2.40-1.fc40.noarch 37/41 Verifying : cri-tools-1.26.0-150500.1.1.x86_64 38/41 Verifying : kubeadm-1.24.17-150500.1.1.x86_64 39/41 Verifying : kubectl-1.24.17-150500.1.1.x86_64 40/41 Verifying : kubelet-1.24.17-150500.1.1.x86_64 41/41 Installed: conntrack-tools-1.4.7-3.fc39.x86_64 containernetworking-plugins-1.3.0-3.fc39.x86_64 cri-tools-1.26.0-150500.1.1.x86_64 cryptsetup-libs-2.6.1-3.fc39.x86_64 dbus-1:1.14.10-1.fc40.x86_64 dbus-broker-33-2.fc39.x86_64 dbus-common-1:1.14.10-1.fc40.noarch device-mapper-1.02.196-1.fc39.x86_64 device-mapper-libs-1.02.196-1.fc39.x86_64 diffutils-3.10-3.fc39.x86_64 ebtables-legacy-2.0.11-14.fc39.x86_64 ethtool-2:6.5-1.fc40.x86_64 iproute-6.5.0-1.fc40.x86_64 iptables-legacy-1.8.10-2.fc40.x86_64 iptables-legacy-libs-1.8.10-2.fc40.x86_64 iptables-libs-1.8.10-2.fc40.x86_64 kmod-libs-30-6.fc39.x86_64 kubeadm-1.24.17-150500.1.1.x86_64 kubectl-1.24.17-150500.1.1.x86_64 kubelet-1.24.17-150500.1.1.x86_64 libargon2-20190702-3.fc39.x86_64 libbpf-2:1.2.0-1.fc40.x86_64 libfdisk-2.39.2-1.fc40.x86_64 libmnl-1.0.5-3.fc39.x86_64 libnetfilter_conntrack-1.0.9-2.fc39.x86_64 libnetfilter_cthelper-1.0.0-24.fc39.x86_64 libnetfilter_cttimeout-1.0.0-22.fc39.x86_64 libnetfilter_queue-1.0.5-5.fc39.x86_64 libnfnetlink-1.0.1-24.fc39.x86_64 libseccomp-2.5.3-6.fc39.x86_64 libutempter-1.2.1-10.fc39.x86_64 libxkbcommon-1.6.0-1.fc40.x86_64 psmisc-23.6-4.fc39.x86_64 qrencode-libs-4.1.1-5.fc39.x86_64 socat-1.7.4.4-3.fc39.x86_64 systemd-254.5-2.fc40.x86_64 systemd-networkd-254.5-2.fc40.x86_64 systemd-pam-254.5-2.fc40.x86_64 systemd-resolved-254.5-2.fc40.x86_64 util-linux-2.39.2-1.fc40.x86_64 xkeyboard-config-2.40-1.fc40.noarch Complete! ```
neolit123 commented 1 year ago

this is not a kubeadm bug. the k8s.io maintainers might be able to help you.

/kind support /transfer k8s.io

nitinjagjivan commented 1 year ago

Thanks @neolit123 An interesting discovery is that it works well when I install Kubernetes version 1.24.0 on a fresh AWS instance. In a previous instance, I successfully installed version 1.28.3, uninstalled it, and then attempted to install version 1.24.0 and got this error.

nitinjagjivan commented 1 year ago

I think the issue is yum cache. I need to refresh the package cache in YUM before installing kubeadm.

sudo yum clean all
sudo yum makecache
sudo yum update