haminhcong / openstack-til

0 stars 0 forks source link

Networking Problem when Install Docker on Bridge-Veth Host #1

Closed haminhcong closed 3 years ago

haminhcong commented 3 years ago

With this configuration:

# /etc/systemd/network/25-veth-0-1.netdev
[NetDev]
Name=veth0
Kind=veth

[Peer]
Name=veth1
# /etc/systemd/network/25-veth-2-3.netdev
[NetDev]
Name=veth2
Kind=veth

[Peer]
Name=veth3
# /etc/netplan/00-installer-config.yaml
network:
  version: 2
  renderer: networkd
  ethernets:
    ens33:
      dhcp4: false
      dhcp6: false
    veth0: {}
    veth1:
      addresses:
        - 192.168.175.11/24
      gateway4: 192.168.175.2
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]
    veth2: {}
    veth3: {}  
  bridges:
    br-ens33:
      interfaces: 
        - ens33
        - veth0
        - veth2
      dhcp4: false        
      dhcp6: false

when start docker, networking is lost

and with this configuration


cat /etc/netplan/00-installer-config.yaml 
# This is the network config written by 'subiquity'
network:
  version: 2
  renderer: networkd
  ethernets:
    ens33:
      dhcp4: false
      dhcp6: false
    veth0: {}
    veth1: {}
    veth2: {}
    veth3: {}  
  bridges:
    br-ens33:
      addresses:
        - 192.168.175.11/24
      gateway4: 192.168.175.2
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]

      interfaces: 
        - ens33
        - veth0
        - veth2
      dhcp4: false

when start docker, networking is not lost

haminhcong commented 3 years ago

Server network config after start docker service (In case not lost connection)

ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-ens33 state UP group default qlen 1000
    link/ether 00:0c:29:e2:1a:1a brd ff:ff:ff:ff:ff:ff
3: veth3@veth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ea:1a:1e:9b:b6:7b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::e81a:1eff:fe9b:b67b/64 scope link 
       valid_lft forever preferred_lft forever
4: veth2@veth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-ens33 state UP group default qlen 1000
    link/ether 8a:99:2c:2a:ac:7d brd ff:ff:ff:ff:ff:ff
5: veth1@veth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ee:b8:0e:2f:25:6d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::ecb8:eff:fe2f:256d/64 scope link 
       valid_lft forever preferred_lft forever
6: veth0@veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-ens33 state UP group default qlen 1000
    link/ether 76:f0:5c:fe:1b:98 brd ff:ff:ff:ff:ff:ff
7: br-ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether de:91:92:bd:3d:e4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.175.11/24 brd 192.168.175.255 scope global br-ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::dc91:92ff:febd:3de4/64 scope link 
       valid_lft forever preferred_lft forever
8: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:1a:1c:36:85 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:1aff:fe1c:3685/64 scope link 
       valid_lft forever preferred_lft forever

brctl show

bridge name bridge id       STP enabled interfaces
br-ens33        8000.de9192bd3de4   no      ens33
                            veth0
                            veth2
docker0     8000.02421a1c3685   no  
haminhcong commented 3 years ago

Problem Reason

After docker daemon started, it insert this rule to host iptables:

-P FORWARD DROP

this rule DROP all packet go from/to bridge br-ens33

Solution

From idea in another Docker iptables rules

-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT

Add following iptables rules to allow packet forwarding on br-ens33 bridge

iptables -A FORWARD -p all -i br-ens33 -j ACCEPT
iptables -A FORWARD -p all -o br-ens33 -j ACCEPT

after add two above rules, host networking on ip 192.168.175.11/24 is working normally again.