search

kubernetes / kubeadm

Aggregator for issues filed against kubeadm
Apache License 2.0
3.75k stars 713 forks source link

kubeadm complains about bridge-nf-call and ip_forward if not using docker runtime #1062

Closed sysrich closed 6 years ago

sysrich commented 6 years ago

Is this a BUG REPORT or FEATURE REQUEST?:

/kind bug

What happened:

After booting a clean system, running kubeadm init with a CRI runtime other than docker configured produces the following pair of error messages:

    [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables does not exist
    [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1

What you expected to happen:

Those two pre-flight checks to pass because they could be satisfied automatically.

Obviously, these are handled automatically by the starting of the docker daemon normally, but for openSUSE Kubic we are investigating using CRI-O by default, where we do not have the luxury of an overbearing daemon to meddle with such things.

Therefore this bug report is either an opportunity for kubeadm to handle things itself. I think there is a case to be that because kubeadm knows what it needs, kubeadm should modprobe br_netfilter and echo '1' > /proc/sys/net/ipv4/ip_forward automatically rather than complaining about the issues.

Alternatively, if this suggestion is not acceptable, I'd appreciate a suggestion as to how openSUSE Kubic should automatically address these issues in a way that will remain aligned with kubeadm's general expectations.

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

outcoldman commented 6 years ago

Does not feel like a product bug, but documentation bug (just a missing manual). Current we have only pages on how to install kubernetes with kubeadm with docker as container runtime, we should have another one for cri-o.

Vogtinator commented 6 years ago

IMO kubeadm should at least do the modprobe itself. Whether it's the case for ip_forward as well is debatable as that is system configuration and up to the admin.

sysrich commented 6 years ago

@vrothberg - what do you think? Should the above 2 conditions (modprobe and sysctl) be rectified automatically by kubeadm, or do you think this is something better handled in CRI-O?

Docker does both automagically itself

And if you think CRI-O proper shouldn't take care, where do you think the dirty hack should be carried in openSUSE? in the cri-o package or in the kubeadm package? ;)

vrothberg commented 6 years ago

I think that’s something kubeadm package should do. After that, we can check if that’s really an error or if it can be demoted to an info log. Many things in K8s are still built around how Docker does things but they’re not always necessary.

I am on vacation at the moment but will check mails here and then. Thanks for the ping.

On Wed 12. Sep 2018 at 10:30, Richard Brown notifications@github.com wrote:

@vrothberg https://github.com/vrothberg - what do you think? Should the above 2 conditions (modprobe and sysctl) be rectified automatically by kubeadm, or do you think this is something better handled in CRI-O?

Docker does both automagically itself

And if you think CRI-O proper shouldn't take care, where do you think the dirty hack should be carried in openSUSE? in the cri-o package or in the kubeadm package? ;)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/kubeadm/issues/1062#issuecomment-420559942, or mute the thread https://github.com/notifications/unsubscribe-auth/ALI4g1dRKf5vWXz_7H27VktuD8nP5tAsks5uaMYogaJpZM4V_W70 .

sysrich commented 6 years ago

This issue will be handled in openSUSE with the following changes: https://build.opensuse.org/package/rdiff/devel:kubic/kubernetes?linkrev=base&rev=9

I'm planning on submitting something similar in the upstream rpm packaging promptly

itpings commented 5 years ago

Restarting Docker did the trick...Thanks you

adobo76 commented 5 years ago

I second adding this information to the setup page. I was seeing this error and it caused me at least 20 min of pain until I came across this thread. Thank you for sharing the solution.

danielsand commented 4 years ago

make sure that you follow the procedure then the error doesnt occour https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cri-o

natemoseman commented 4 years ago

simply modprobe'ng things doesn't ensure they are running. If you reboot then everything breaks. It should tell you to make sure these things are enabled persistently. Which is does for the sysctl rules, but not the modules.

When following the directions on stuff like this I don't like to read into things and try to take extra steps based on hunches. If it tells me to do something, I do it. If it doesn't I don't. I wait for things to break and go back and fix them. This way I have a better idea if it's bad documentation that is the problem instead of some random thing I did when things break.

jefflill commented 3 years ago

I probably need to rebuild my local Hyper-V node images.

antaloala commented 2 years ago

I just fresh installed a k8s 1.24 cluster with containerd as runtime engine, using latest kubeadm tooling and following step by step the installation procedure described in latest (1.24) k8s documentation; I experienced the problem reported in this issue when raising kubeadm init and kubeadm join commands. I solved the problem invoking (in master and worker nodes) the commands described at the beginning of this issue

modprobe br_netfilter and echo '1' > /proc/sys/net/ipv4/ip_forward

So I understand this has not been solved yet ...

P.S: It was an installation on Ubuntu-based Virtualbox VMs on a Windows10 host OS where Hyper-V has been fully disabled, so experienced issue is not related to any Hyper-V collision (not in my case).

neolit123 commented 2 years ago

See https://kubernetes.io/docs/setup/production-environment/container-runtimes/#install-and-configure-prerequisites these are still required when setting up a CR that is k8s compatible.

antaloala commented 2 years ago

Thanks @neolit123 (yes, it is all there .. and I did not realize on it :-(

RicHincapie commented 2 years ago

I just fresh installed a k8s 1.24 cluster with containerd as runtime engine, using latest kubeadm tooling and following step by step the installation procedure described in latest (1.24) k8s documentation; I experienced the problem reported in this issue when raising kubeadm init and kubeadm join commands. I solved the problem invoking (in master and worker nodes) the commands described at the beginning of this issue

modprobe br_netfilter and echo '1' > /proc/sys/net/ipv4/ip_forward

So I understand this has not been solved yet ...

P.S: It was an installation on Ubuntu-based Virtualbox VMs on a Windows10 host OS where Hyper-V has been fully disabled, so experienced issue is not related to any Hyper-V collision (not in my case).

The same pre-flight erros happened to me with kubeadm, cri-o, and kubelet on 1.22

For any one looking for a fast work around, this is the answer.

syamprasadsiddabattula commented 1 year ago

thank u sir.......i am happy with ur solutions sir....tQ

Napsty commented 1 year ago

To make this boot-resistant, add br_netfilter to the list of kernel modules to load at boot time:

echo "br_netfilter" | sudo tee -a /etc/modules

Or into a dedicated file:

echo "br_netfilter" | sudo tee /etc/modules-load.d/netfilter.conf

This assumes you are using a Linux distribution with Systemd init, systemd-modules-load.service is responsible for loading these mentioned modules.

WillNilges commented 1 year ago

Still getting the ip_forward error on Raspberry Pi OS (Debian 11) and Kube 1.26

        [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1

Anyone got a boot-resistant solution for that one?

pacoxu commented 1 year ago

An example from kubespray: https://github.com/kubernetes-sigs/kubespray/blob/36e5d742dc2b3f7984398c38009f236be7c3c065/Vagrantfile#L224.

Another example in kubeadm blog: https://kubernetes.io/blog/2020/02/07/deploying-external-openstack-cloud-provider-with-kubeadm/#install-docker-and-kubernetes

samnu88 commented 6 months ago

try modprobe br_netfilter echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables echo 1 > /proc/sys/net/ipv4/ip_forward