search

microsoft / WSL

Issues found on WSL
https://docs.microsoft.com/windows/wsl
MIT License
17.4k stars 819 forks source link

Adding nftables sets doesn't work on WSL 2 #6044

Open EnGyUncia opened 4 years ago

EnGyUncia commented 4 years ago

Environment

Microsoft Windows [Version 10.0.19041.508] Ubuntu 20.04 WSL 2

Steps to reproduce

After installing nftables (my version from repo: 0.9.3) and trying to configure named set in specified table - i'm getting an error.

$ sudo nft add table inet test_table
$ sudo nft add chain inet test_table test_chain {type filter hook input priority 0\;}
$ sudo nft add set inet test_table my_set {type ipv4_addr\;}
Error: Could not process rule: Operation not supported
add set inet test_table my_set {type ipv4_addr;}
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expected behavior

The set must be created and displayed in listing (nft list ruleset)

Actual behavior

I got the following error: "Could not process rule: Operation not supported"

GitHub_issue

zyablitsev commented 4 years ago

I also have the same issue.

Environment Microsoft Windows [Version 10.0.19041.572] Debian 10 WSL 2

therealkenc commented 4 years ago

image

therealkenc commented 3 years ago

There are some others vs. Ububtu's expected kconfig:

CONFIG_NF_TABLES_SET
CONFIG_NF_TABLES_NETDEV
CONFIG_NF_TABLES_ARP
CONFIG_NF_TABLES_BRIDGE

image

kobenauf commented 2 years ago

Is this being addressed? I can't run native docker without reverting back to legacy iptables, which I can imagine is not going to be around forever. What is a longer-term path to run native docker? Ticket 6655, which was closed as a dup of this, describes the symptoms/effect very well.

lygstate commented 2 years ago

ping for this.

biwiki commented 2 years ago

I have the same issue with Jammy

panktrip commented 2 years ago

I am facing same issue in ubuntu-22.04 Jammy release. Docker does not start here, due to below error.

failed to start daemon: Error initializing network controller: error obtaining controller instance: unable to add return rule in DOCKER-ISOLATION-STAGE-1 chain: (iptables failed: iptables --wait -A DOCKER-ISOLATION-STAGE-1 -j RETURN: iptables v1.8.7 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain DOCKER-ISOLATION-STAGE-1 (exit status 4))

kobenauf commented 2 years ago

@panktrip I had to do something similar to what was referenced in another ticket.

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy
panktrip commented 2 years ago

Thanks a lot @kobenauf . This worked for me on Ubuntu-22.04 in WSL2.

crbeckle commented 2 years ago

If you are running into this problem from an RPM-based distribution, many of the fixes given may not work for you. Specifically, I ran into this issue using the AlmaLinux-8 distro running under distrod (to enable the use of systemd). I originally went down the path of trying to find and install the legacy version of iptables (xtables-legacy-multi) to use the same fix mentioned above; however, after spending many hours in (unsuccessfully) trying to find that package in non-Fedora yum repositories, I pivoted to fixing the issue in the kernel instead.

As referenced by @therealkenc, the issue of supporting the newer, nftables-based iptables is that the proper interfaces are not available in the standard WSL kernel; however, the interfaces exist in the code - they just need to be configured on / compiled in. Here is how I went about fixing the issue in AlmaLinux-8 (under distrod) starting from a fresh install:

  1. Install requisite packages for building the kernel (thanks to this guide for pointing me in the right direction)
    1. sudo dnf config-manager --set-enabled powertools
    2. sudo dnf group install "Development Tools"
    3. sudo dnf install ncurses-devel binutils-devel openssl-devel python39 dwarves
  2. Clone the kernel repo: git clone https://github.com/microsoft/WSL2-Linux-Kernel.git kernel (note that the default checkout branch is probably what you will need, but run uname -r and make sure the version of kernel you have corresponds to the branch you are checked out to in the repo)
  3. Configure the kernel build
    1. cd kernel
    2. cp Microsoft/config-wsl .config
    3. make menuconfig
    4. At this point, you are presented with a dialog text UI (TUI) that allows you to configure which modules are built into the kernel. Descend down into Networking Support → Networking options → Network packet filtering framework (Netfilter), and find any option having to do with nf_tables and enable it (including under the Core Netfilter Configuration sub-menu). I readily admit that I do not know which subset of these options is actually required for iptables using nf_tables to work, so I just blanket enabled them all.
    5. (Optional) Exit back up to the top-level menu and go into General setup → Local version - append to kernel release, then change the name to something that reflects the kernel modifications (this is just to easily keep track of what kernel you are using at runtime)
    6. Save and Exit back out of the menus
  4. Build the kernel: make KCONFIG_CONFIG=.config -j $(nproc)
  5. Copy the kernel outside of the VM: cp arch/x86/boot/bzImage /mnt/c/Users/<YOUR USER>/kernel
  6. Follow the instructions for global WSL configuration to point to the kernel file you just copied
  7. Shutdown WSL: wsl --shutdown
  8. Re-launch your distro

At this point, if you performed the optional step to give a custom kernel postfix, you can run uname -r to ensure you are using the new kernel. If so, then try restarting the docker daemon (i.e. sudo systemctl restart docker) and confirm that it is up and running.

I realize this is a heavy-handed approach, but I hope it can help any folks using an RPM-based distribution until the problem is officially addressed either by docker or by the WSL standard kernel.

Secondly, I realize that RPM-based distributions have moved towards the use of podman, but I have seen reports of similar issues with iptables, especially when using sudo podman, so hopefully this can help in that scenario as well. As a side note, if you want to be able to install both podman and docker at the same time in AlmaLinux-8 (for example, to play nicely with Jenkins pipelines or if podman features are unstable), you can follow this guide to do so.

hluengas commented 2 years ago

If you are running into this problem from an RPM-based distribution, many of the fixes given may not work for you. Specifically, I ran into this issue using the AlmaLinux-8 distro running under distrod (to enable the use of systemd). I originally went down the path of trying to find and install the legacy version of iptables (xtables-legacy-multi) to use the same fix mentioned above; however, after spending many hours in (unsuccessfully) trying to find that package in non-Fedora yum repositories, I pivoted to fixing the issue in the kernel instead.

As referenced by @therealkenc, the issue of supporting the newer, nftables-based iptables is that the proper interfaces are not available in the standard WSL kernel; however, the interfaces exist in the code - they just need to be configured on / compiled in. Here is how I went about fixing the issue in AlmaLinux-8 (under distrod) starting from a fresh install:

1. Install requisite packages for building the kernel (thanks to [this guide](https://kumekay.com/compiling-custom-kernel-for-wsl2/) for pointing me in the right direction)

   1. `sudo dnf config-manager --set-enabled powertools`
   2. `sudo dnf group install "Development Tools"`
   3. `sudo dnf install ncurses-devel binutils-devel openssl-devel python39 dwarves`

2. Clone the kernel repo: `git clone https://github.com/microsoft/WSL2-Linux-Kernel.git kernel` (note that the default checkout branch is probably what you will need, but run `uname -r` and make sure the version of kernel you have corresponds to the branch you are checked out to in the repo)

3. Configure the kernel build

   1. `cd kernel`
   2. `cp Microsoft/config-wsl .config`
   3. `make menuconfig`
   4. At this point, you are presented with a `dialog` text UI (TUI) that allows you to configure which modules are built into the kernel. Descend down into **Networking Support** → **Networking options** → **Network packet filtering framework (Netfilter)**, and find any option having to do with nf_tables and enable it (including under the **Core Netfilter Configuration** sub-menu). I readily admit that I do _not_ know which subset of these options is actually required for iptables using nf_tables to work, so I just blanket enabled them all.
   5. (Optional) Exit back up to the top-level menu and go into **General setup** → **Local version - append to kernel release**, then change the name to something that reflects the kernel modifications (this is just to easily keep track of what kernel you are using at runtime)
   6. Save and Exit back out of the menus

4. Build the kernel: `make KCONFIG_CONFIG=.config -j $(nproc)`

5. Copy the kernel outside of the VM: `cp arch/x86/boot/bzImage /mnt/c/Users/<YOUR USER>/kernel`

6. Follow the [instructions for global WSL configuration](https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configuration-setting-for-wslconfig) to point to the kernel file you just copied

7. Shutdown WSL: `wsl --shutdown`

8. Re-launch your distro

At this point, if you performed the optional step to give a custom kernel postfix, you can run uname -r to ensure you are using the new kernel. If so, then try restarting the docker daemon (i.e. sudo systemctl restart docker) and confirm that it is up and running.

I realize this is a heavy-handed approach, but I hope it can help any folks using an RPM-based distribution until the problem is officially addressed either by docker or by the WSL standard kernel.

Secondly, I realize that RPM-based distributions have moved towards the use of podman, but I have seen reports of similar issues with iptables, especially when using sudo podman, so hopefully this can help in that scenario as well. As a side note, if you want to be able to install both podman and docker at the same time in AlmaLinux-8 (for example, to play nicely with Jenkins pipelines or if podman features are unstable), you can follow this guide to do so.

This worked for me running Rocky Linux in WSL2. Can't thank you enough, it's been causing me so many issues!

wonson commented 1 year ago

@panktrip I had to do something similar to what was referenced in another ticket.

update-alternatives --set iptables /usr/sbin/iptables-legacy
update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy

This does not really solved the problem. Even though you get a pass for matching comment here, you may still hit error when matching conntrack

pettermk commented 1 year ago

If you are running into this problem from an RPM-based distribution, many of the fixes given may not work for you. Specifically, I ran into this issue using the AlmaLinux-8 distro running under distrod (to enable the use of systemd). I originally went down the path of trying to find and install the legacy version of iptables (xtables-legacy-multi) to use the same fix mentioned above; however, after spending many hours in (unsuccessfully) trying to find that package in non-Fedora yum repositories, I pivoted to fixing the issue in the kernel instead.

As referenced by @therealkenc, the issue of supporting the newer, nftables-based iptables is that the proper interfaces are not available in the standard WSL kernel; however, the interfaces exist in the code - they just need to be configured on / compiled in. Here is how I went about fixing the issue in AlmaLinux-8 (under distrod) starting from a fresh install:

1. Install requisite packages for building the kernel (thanks to [this guide](https://kumekay.com/compiling-custom-kernel-for-wsl2/) for pointing me in the right direction)

   1. `sudo dnf config-manager --set-enabled powertools`
   2. `sudo dnf group install "Development Tools"`
   3. `sudo dnf install ncurses-devel binutils-devel openssl-devel python39 dwarves`

2. Clone the kernel repo: `git clone https://github.com/microsoft/WSL2-Linux-Kernel.git kernel` (note that the default checkout branch is probably what you will need, but run `uname -r` and make sure the version of kernel you have corresponds to the branch you are checked out to in the repo)

3. Configure the kernel build

   1. `cd kernel`
   2. `cp Microsoft/config-wsl .config`
   3. `make menuconfig`
   4. At this point, you are presented with a `dialog` text UI (TUI) that allows you to configure which modules are built into the kernel. Descend down into **Networking Support** → **Networking options** → **Network packet filtering framework (Netfilter)**, and find any option having to do with nf_tables and enable it (including under the **Core Netfilter Configuration** sub-menu). I readily admit that I do _not_ know which subset of these options is actually required for iptables using nf_tables to work, so I just blanket enabled them all.
   5. (Optional) Exit back up to the top-level menu and go into **General setup** → **Local version - append to kernel release**, then change the name to something that reflects the kernel modifications (this is just to easily keep track of what kernel you are using at runtime)
   6. Save and Exit back out of the menus

4. Build the kernel: `make KCONFIG_CONFIG=.config -j $(nproc)`

5. Copy the kernel outside of the VM: `cp arch/x86/boot/bzImage /mnt/c/Users/<YOUR USER>/kernel`

6. Follow the [instructions for global WSL configuration](https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configuration-setting-for-wslconfig) to point to the kernel file you just copied

7. Shutdown WSL: `wsl --shutdown`

8. Re-launch your distro

At this point, if you performed the optional step to give a custom kernel postfix, you can run uname -r to ensure you are using the new kernel. If so, then try restarting the docker daemon (i.e. sudo systemctl restart docker) and confirm that it is up and running.

I realize this is a heavy-handed approach, but I hope it can help any folks using an RPM-based distribution until the problem is officially addressed either by docker or by the WSL standard kernel.

Secondly, I realize that RPM-based distributions have moved towards the use of podman, but I have seen reports of similar issues with iptables, especially when using sudo podman, so hopefully this can help in that scenario as well. As a side note, if you want to be able to install both podman and docker at the same time in AlmaLinux-8 (for example, to play nicely with Jenkins pipelines or if podman features are unstable), you can follow this guide to do so.

I can confirm this fixes similar issues with podman, where port forwarding was essentially broken, in WSL2, Oracle Linux. Thanks ever so much 🙏

Would be great to have this fixed in the default kernel though!

sreemtech commented 8 months ago

Docker and WSL2 : Rocky Linux 9 iptables -A test0-in -p tcp --dport 1111 -m state --state NEW -m recent --set Warning: Extension recent revision 0 not supported, missing kernel module? iptables v1.8.8 (nf_tables): RULE_APPEND failed (No such file or directory): rule in chain

Any Advise please ?

SuperSandro2000 commented 7 months ago

When activating nftables on NixOS I got a No such file or directory error on this line fib saddr . mark . iif oif exists accept and could get it to work by building a custom kernel with the following patch

diff --git a/arch/x86/configs/config-wsl b/arch/x86/configs/config-wsl
index 51935f9f2..365529760 100644
--- a/arch/x86/configs/config-wsl
+++ b/arch/x86/configs/config-wsl
@@ -1066,7 +1069,7 @@ CONFIG_NF_NAT_MASQUERADE=y
 CONFIG_NETFILTER_SYNPROXY=y
 CONFIG_NF_TABLES=y
 CONFIG_NF_TABLES_INET=y
-# CONFIG_NF_TABLES_NETDEV is not set
+CONFIG_NF_TABLES_NETDEV=y
 CONFIG_NFT_NUMGEN=y
 CONFIG_NFT_CT=y
 CONFIG_NFT_COUNTER=y
@@ -1084,11 +1087,18 @@ CONFIG_NFT_REJECT=y
 CONFIG_NFT_REJECT_INET=y
 CONFIG_NFT_COMPAT=y
 # CONFIG_NFT_HASH is not set
+CONFIG_NFT_FIB=y
+CONFIG_NFT_FIB_INET=y
 CONFIG_NFT_XFRM=y
 CONFIG_NFT_SOCKET=y
 # CONFIG_NFT_OSF is not set
 # CONFIG_NFT_TPROXY is not set
 # CONFIG_NFT_SYNPROXY is not set
+# CONFIG_NF_DUP_NETDEV is not set
+# CONFIG_NFT_DUP_NETDEV is not set
+# CONFIG_NFT_FWD_NETDEV is not set
+CONFIG_NFT_FIB_NETDEV=y
+# CONFIG_NFT_REJECT_NETDEV is not set
 # CONFIG_NF_FLOW_TABLE is not set
 CONFIG_NETFILTER_XTABLES=y
 # CONFIG_NETFILTER_XTABLES_COMPAT is not set
@@ -1256,7 +1266,7 @@ CONFIG_NF_SOCKET_IPV4=y
 CONFIG_NF_TABLES_IPV4=y
 CONFIG_NFT_REJECT_IPV4=y
 # CONFIG_NFT_DUP_IPV4 is not set
-# CONFIG_NFT_FIB_IPV4 is not set
+CONFIG_NFT_FIB_IPV4=y
 # CONFIG_NF_TABLES_ARP is not set
 # CONFIG_NF_DUP_IPV4 is not set
 # CONFIG_NF_LOG_ARP is not set
@@ -1295,7 +1305,7 @@ CONFIG_NF_SOCKET_IPV6=y
 CONFIG_NF_TABLES_IPV6=y
 CONFIG_NFT_REJECT_IPV6=y
 # CONFIG_NFT_DUP_IPV6 is not set
-# CONFIG_NFT_FIB_IPV6 is not set
+CONFIG_NFT_FIB_IPV6=y
 # CONFIG_NF_DUP_IPV6 is not set
 CONFIG_NF_REJECT_IPV6=y
 CONFIG_NF_LOG_IPV6=y