DHCP replaced and as a consequence DNS not working

[ghost]

[Documentation Enhancement Request]

Using firejail in the context of TorMiddlebox, I ran into this issue.

Pointers : https://github.com/Bylon/TOR_Middlebox/wiki/Firefox---Middlebox-tricks Bylon/TOR_Middlebox#4

From my investigations, contrary to a VM that does a regular DHCP and gets all the correct network settings, firejail tries to "guess" things and the result is not working.

To summarize the TorMiddlebox settings, we have a bridged interface, vnet0 declared as such

$ cat /etc/network/interfaces
iface lo inet loopback

# VirtualBox interface for TOR middlebox
auto vnet0
iface vnet0 inet static
  address 172.16.0.1
  netmask 255.255.255.0
  bridge_ports none
  bridge_maxwait 0

In the Ubuntu setting, we have NetworkManager that uses dnsmasq-base which gets the additional parameters as such:

$ cat /etc/NetworkManager/dnsmasq.d/middlebox.conf 
# Modifications for dnsmasq spawned by NM
# in order to make it work for our middlebox.

# Or which to listen on by address (remember to include 127.0.0.1 if
# you use this.)
listen-address=172.16.0.1

# Uncomment this to enable the integrated DHCP server, you need
# to supply the range of addresses available for lease and optionally
# a lease time. If you have more than one network, you will need to
# repeat this for each network on which you want to supply DHCP
# service.
dhcp-range=vnet0,172.16.0.2,172.16.0.254,1h

dnsmasq is then listening to the above interface (vnet0) in addition to the loopback address, as we see here:

$ sudo netstat -tulpn | grep dnsmasq
tcp        0      0 127.0.1.1:53            0.0.0.0:*               LISTEN      2053/dnsmasq    
tcp        0      0 172.16.0.1:53           0.0.0.0:*               LISTEN      2053/dnsmasq    
udp        0      0 0.0.0.0:52185           0.0.0.0:*                           2053/dnsmasq    
udp        0      0 127.0.1.1:53            0.0.0.0:*                           2053/dnsmasq    
udp        0      0 172.16.0.1:53           0.0.0.0:*                           2053/dnsmasq    
udp        0      0 0.0.0.0:67              0.0.0.0:*                           2053/dnsmasq    

All is well, dnsmasq serves both DNS requests (53) and DCHP requests (67)

In fact, there is an "upstream" DNS obviously, which in my configuration is given by my ISP's box, as we see with nmcli.

$ nmcli dev show enp0s31f6 
(...)
IP4.DNS[1]:                             192.168.0.254

This is where dnsmasq gets the DNS resolution and caches it.

If we use the "tcpdump trick" described in this post: https://askubuntu.com/questions/637893/how-to-know-what-dns-am-i-using-in-ubuntu-from-14-04-onwards/637894#637894

from our host, we see that DNS are served by 127.0.1.1,which is where dnsmasq caches DNS requests for our host, and then if you analyse traffic on the ethernet outgoing connection to the ISP you see the actual DNS requests.

For our middlebox, we obviously have iptables rules both for DHCP and DNS. Here is out it looks like:

$ sudo iptables -L -v -n --line-numbers; sudo iptables -L -v -n --line-numbers -t nat
Chain INPUT (policy ACCEPT 2472K packets, 35G bytes)
num   pkts bytes target     prot opt in     out     source               destination         
(...)
5        7  2318 ACCEPT     udp  --  vnet0  *       0.0.0.0/0            0.0.0.0/0            udp dpt:67
9       77  5171 ACCEPT     udp  --  vnet0  *       172.16.0.0/24        172.16.0.1           udp dpt:60

Chain OUTPUT (policy ACCEPT 2135K packets, 117M bytes)
num   pkts bytes target     prot opt in     out     source               destination         
(...)
5        7  2301 ACCEPT     udp  --  *      vnet0   172.16.0.1           172.16.0.0/24        udp spt:67 dpt:68
9       77  5987 ACCEPT     udp  --  *      vnet0   172.16.0.1           172.16.0.0/24        udp spt:60

Chain PREROUTING (policy ACCEPT 92 packets, 12606 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
(...)
3       77  5171 REDIRECT   udp  --  vnet0  *       172.16.0.0/24        0.0.0.0/0            udp dpt:53 redir ports 60

We redirect UDP-53 from vnet0 to port 60, that is where Tor is listening to resolve DNS requests.

So, now lets look at how VirtualBox does things when we start a VM attached to our vnet0 interface.

Once the guest starts, it issues a DHCP Discover (broadcast). Since we have dnsmasq listening for DHCP on vnet0, we get a DHCP Offer with the right parameters: the offer is made by 172.16.0.1 (where dnsmasq listens) that also sets itself as the gateway and the DNS server (plus the usual IPV4 address, lease renewal, rebiding, etc...)

Then, when the VM guests wants to resolve a host name, it issues a DNS request on 172.16.0.1, which will go through tor in our configuration (but could also be served by our ISP's box in a "regular" setting).

Now let's look at what firejail does when attached to the same vnet0 bridged interface. Start wireshark filtering UDP on that interface, then: firejail --net=vnet0 --noprofile

As you can see, unlike VirtualBox, there is no DHCP request at all showing on wireshark. Doing an ifconfig inside this new shell shows that it managed to get an IPV4 address:

$ ifconfig
eth0      Link encap:Ethernet  HWaddr d6:25:7f:e2:4a:07  
          inet adr:172.16.0.46  Bcast:172.16.1.255  Masque:255.255.255.0

But when using the same "tcpdump trick" as above inside this firejail guest, we see that it tries to use, same as the host, 127.0.1.1 as DNS server.

This obviously is a nonsense since we are in a network sandbox, nobody is responding to UDP-53 on 127.0.1.1 inside the sandbox, hence DNS fail from the sandbox.

Workaround: As described in the ticket on TorMiddlebox in introduction, the workaround is easy. Since firejail makes a "guess" that does not work on what is the real DNS server, we just need to add a parameter to force the right value, which is then:

firejail --net=vnet0 --dns=172.16.0.1

Enhancement: Not sure whether that would be possible (I have not investigated) but since firejail creates a new network environment (when instructed to), why wouldn't it issue a plain regular DHCP request instead of trying to "guess" things? Or maybe add an option that instructs to issue a DHCP instead of the current way of setting the network configuration?

Note: I have also tried with the last firejail version from this github, instead of the LTS we have in 16.04, it does not change anything about this enhancement request.

[startx2017]

But when using the same "tcpdump trick" as above inside this firejail guest, we see that it tries to use, same as the host, 127.0.1.1 as DNS server.

We reuse the host DNS configuration inside the sandbox. In your case it was 127.0.1.1 on the host, but inside the sandbox nobody is listening on 127.0.1,1, so you need to send it somewhere else using --dns=... command.

why wouldn't it issue a plain regular DHCP request instead of trying to "guess" things?

Running a DHCP client inside the sandbox has some security implications. The DHCP server has to much control over your sandbox: it can decide to change you IP address, your DNS settings, and so on.

[ghost]

Thanks for your response.

The security implications running a DHCP are perfectly clear!

I was just wondering, since setting DNS inside the sandbox to 127.0.1.1 is obviously useless, whether the DNS "guess" couldn't be better. In my setting, since there is a listener on the network (vnet0) we put the sandbox on, this ip address (172.16.0.1) belonging to the network could be elected as better suited than 127.0.1.1 that leads nowhere.

I agree though, that since 127.0.1.1 gives the user a DNS that does not work, it will make him think of what proper DNS it can offer according to the security he has in mind.

I still think that it could be best documented (for example in the --dns option) so that the user does not search for hours what he did wrong and knows immediately he must provide a working dns.

So, suggestions, based on your response:

• work as design (nothing to do in the code)
• improve documentation (explain: you don't call DHCP, you use the same DNS as the host that could be inoperative when it is 127.0.1.1, in which case the user MUST provide a working DNS with the --dns option, should he need DNS resolution)
• close the ticket

Thanks.

[ghost]

I changed the first post accordingly to: Documentation Enhancement Request.

(Instead of: Enhancement Request)

[netblue30]

Yes, we have to document it!

[chiraag-nataraj]

@netblue30 Have we documented this now?