search

canonical / lxd

Powerful system container and virtual machine manager
https://canonical.com/lxd
GNU Affero General Public License v3.0
4.38k stars 931 forks source link

Failed adding outbound NAT rules for bridge network #11043

Closed jlutran closed 2 years ago

jlutran commented 2 years ago

Required information

Issue description

Lxc bridged network creation is crashing during outbound NAT rules setup

Steps to reproduce

  1. snap stop lxd.daemon
  2. lxd --debug --group lxd
  3. lxc network create lxdbr0 --type=bridge --debug

Information to attach

[...]
DEBUG  [2022-10-21T09:38:55+02:00] Sending request to LXD                        etag= method=POST url="http://unix.socket/1.0/networks"
DEBUG  [2022-10-21T09:38:55+02:00]
    {
        "config": {},
        "description": "",
        "name": "lxdbr0",
        "type": "bridge"
    }
Error: Failed to setup firewall: Failed adding outbound NAT rules for network "lxdbr0" (inet): Failed apply nftables config: Failed to run: nft -f -: exit status 1 (/dev/stdin:3:7-18: Error: Could not process rule: Operation not supported
chain pstrt.lxdbr0 {
      ^^^^^^^^^^^^
/dev/stdin:3:7-18: Error: Could not process rule: No such file or directory
chain pstrt.lxdbr0 {
      ^^^^^^^^^^^^
/dev/stdin:3:7-18: Error: Could not process rule: No such file or directory
chain pstrt.lxdbr0 {
      ^^^^^^^^^^^^)
tomponline commented 2 years ago

This is most likely a missing kernel module for nftables. Probably related to nat given the error message.

What does lsmod show?

jlutran commented 2 years ago

lsmod:

Module                  Size  Used by
bridge                212992  0
stp                    16384  1 bridge
llc                    16384  2 bridge,stp
dm_mod                 98304  0
ebtable_filter         16384  0
ebtables               28672  1 ebtable_filter
ip6table_raw           16384  0
ip6table_mangle        16384  0
ip6table_nat           16384  0
ip6table_filter        16384  0
iptable_raw            16384  0
iptable_mangle         16384  0
iptable_nat            16384  0
nf_nat                 32768  2 iptable_nat,ip6table_nat
nf_conntrack           81920  1 nf_nat
nf_defrag_ipv6         20480  1 nf_conntrack
nf_defrag_ipv4         16384  1 nf_conntrack
iptable_filter         16384  0
nf_tables             180224  0
nfnetlink              16384  1 nf_tables
configfs               32768  1
ip_tables              24576  4 iptable_mangle,iptable_filter,iptable_raw,iptable_nat
tomponline commented 2 years ago

Do you have ipv6 disabled in your system?

jlutran commented 2 years ago

I don't think so:

root@debian:~# cat /sys/module/ipv6/parameters/disable
0
root@debian:~# ip -6 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 2048
    inet6 2a01:cb08:8809:3000:250:43ff:fe03:102/64 scope global dynamic mngtmpaddr
       valid_lft 86353sec preferred_lft 553sec
    inet6 fe80::250:43ff:fe03:102/64 scope link
       valid_lft forever preferred_lft forever
tomponline commented 2 years ago

OK so that maybe the issue then, you're missing a few ipv6 netfilter modules:

For comparison on my system:

lsmod | grep tables
ebtables               45056  1 ebtable_filter
ip6_tables             32768  4 ip6table_filter,ip6table_raw,ip6table_nat,ip6table_mangle
nf_tables             245760  138 nft_masq,nft_chain_nat
nfnetlink              20480  3 nfnetlink_cttimeout,nf_tables
ip_tables              32768  4 iptable_filter,iptable_raw,iptable_nat,iptable_mangle
x_tables               53248  11 ebtables,ip6table_filter,ip6table_raw,iptable_filter,ip6table_nat,ip6_tables,iptable_raw,ip_tables,iptable_nat,ip6table_mangle,iptable_mangle
libcrc32c              16384  5 nf_conntrack,nf_nat,openvswitch,btrfs,nf_tables

lsmod | grep filter
ebtable_filter         16384  0
ebtables               45056  1 ebtable_filter
ip6table_filter        16384  0
ip6_tables             32768  4 ip6table_filter,ip6table_raw,ip6table_nat,ip6table_mangle
iptable_filter         16384  0
bpfilter               16384  0
ip_tables              32768  4 iptable_filter,iptable_raw,iptable_nat,iptable_mangle
x_tables               53248  11 ebtables,ip6table_filter,ip6table_raw,iptable_filter,ip6table_nat,ip6_tables,iptable_raw,ip_tables,iptable_nat,ip6table_mangle,iptable_mangle
jlutran commented 2 years ago

You're right, seems like I missed some kernel modules ! I just ran a new build with this diff:

-# CONFIG_NF_TABLES_INET is not set
+CONFIG_NF_TABLES_INET=y
+CONFIG_NFT_NAT=m
+CONFIG_NFT_REJECT_INET=m
+CONFIG_NFT_FIB=m
-CONFIG_NETFILTER_XTABLES=y
+CONFIG_NETFILTER_XTABLES=m
-# CONFIG_NF_TABLES_IPV4 is not set
+CONFIG_NF_TABLES_IPV4=y
+CONFIG_NFT_REJECT_IPV4=m
+# CONFIG_NFT_DUP_IPV4 is not set
+# CONFIG_NFT_FIB_IPV4 is not set
-# CONFIG_NF_TABLES_IPV6 is not set
+CONFIG_NF_TABLES_IPV6=y
+CONFIG_NFT_REJECT_IPV6=m
+CONFIG_NFT_DUP_IPV6=m
+CONFIG_NFT_FIB_IPV6=m

I'll keep you posted in a new hours

jlutran commented 2 years ago

Issue solved, thanks for your quick replies !

tomponline commented 2 years ago

Excellent thanks!