nftables & firewalld breaks DNS resolution when attached to a custom network

[aaomidi]

Create a new Dockerfile:

Failing dockerfile:

FROM ubuntu:rolling
ENTRYPOINT [ "/sbin/init", "--log-level=err" ]
RUN apt-get update
RUN apt-get install -y systemd openssh-server openssh-server nano vim
RUN echo "root:1" | chpasswd
RUN echo "PermitRootLogin yes" > /etc/ssh/sshd_config
RUN apt-get install -y firewalld dnsutils

Working dockerfile:

FROM ubuntu:focal
ENTRYPOINT [ "/sbin/init", "--log-level=err" ]
RUN apt-get update
RUN apt-get install -y systemd openssh-server openssh-server nano vim
RUN echo "root:1" | chpasswd
RUN echo "PermitRootLogin yes" > /etc/ssh/sshd_config
RUN apt-get install -y firewalld dnsutils

docker build -t firewalld_test .

Create the network:

docker network create -d bridge --subnet 10.6.0.0/16 firewalld_network

Run the container:

docker run --runtime sysbox-runc --ip 10.6.0.3 --rm --network firewalld_network firewalld

SSH into container:

ssh root@10.6.0.3

Run dig:

dig +time=3 +tries=1 @10.6.0.1 google.com # fails
dig +time=3 +tries=1 @1.1.1.1 google.com # works

cat /etc/resolv.conf
nameserver 10.6.0.1 # This is the expected value

[rodnymolina]

@aaomidi, thanks for filing this one up.

Making some notes based on our Slack exchange to make sure we don't miss this ...

Issue seems to be reproduced only in the most recent firewalld release (v0.9.x), which may explain why problem is only seen in Ubuntu 21.10 and CentOS 8.

[aaomidi]

Adding this fixes the problem in ubuntu:rolling

RUN update-alternatives --set iptables /usr/sbin/iptables-legacy

[rodnymolina]

Great, so that proves your previous point that problem seemed to be about "iptables" vs "nf_tables".

If that's the case the fix will need to extend the current iptable-specific logic to address the "nf_tables" scenario.

[aaomidi]

More information:

I changed my host iptables to use iptables-legacy:

sudo update-alternatives --config iptables

And, uh, the broken containers started working fine.

Host	Container	Result
nftable	iptable	Works
iptable	nftable	Works
nftable	nftable	Broken - no DNS resolution
iptable	iptable	Broken - DNS works, no internet connection