netblue30 / firejail

Linux namespaces and seccomp-bpf sandbox
https://firejail.wordpress.com
GNU General Public License v2.0
5.69k stars 557 forks source link

ARP probe failing due to gratuitous arp reply #6133

Open brianvanderburg2 opened 9 months ago

brianvanderburg2 commented 9 months ago

Description

Setup: A bridge is configured with a VLAN subinterface from the main interface bound to it. T When using firejail, the bridge is used as the interface with an IP range to choose from.

enp9s0 (main interface) bm-home (vlan subinterface for home/internet traffic) bhome (bridge, bm-home is placed into this bridge.) no IPs are assigned to the bridges or VLAN/main interfaces

Firejail is launched to create a veth pair to put into bhome, then processes launched under the firejail proccess have an IP/are able to access the internet. The bridge is used to allow other processes/virtual machines/macvlans/etc to also be able to access the internet when desired. This has worked perfectly fine until recently.

Profile:

net bhome
iprange 192.168.1.30,192.168.1.39
defaultgw 192.168.1.1
netmask 255.255.255.0
dns 8.8.8.8
dns 8.8.4.4

Recently, I get an error indicating all the IPs are in use. I checked the wireshark capture and i notice:

Send arp probe I get a gratuitous arp from a different address on the network Send arp probe for next ip same Send arp probe for next ip same

When I don't get the gratuitous arp response, then it assignes that IP address Only one device in our network has that response so I asked them to turn it off, and when they do, it works with no problems.

image

Steps to Reproduce

Unsure. It seems specific to this situation

Expected behavior

After getting no ARP responses matching the IPs of the ARP probe, it should use the IP for the jail

Actual behavior

It performs an ARP probe of two random IPs, then the entire specified range, and fails with the error "Error: cannot assign an IP address; it looks like all of them are in use."

Behavior without a profile

This works, however doesn't use a new network namespace with veth interface connected to the bridge and assigned an iP, so doesn't affect/reflect on this issue

Additional context

Environment

Debian Bullesye Firejail 0.9.72

Checklist

I think the issue is here:

Send ARP probe: image

Received gratuitous ARP image

Code: image

If I'm reading this correctly:

if packet length is to small, continue if proto type is not arp continue copy arp header if arp code is not 2 (reply) continue compare target mac received to probe sender mac, if not the same continue (the probe sender mac is 02:ba:15:00:00:10, the grat arp target mac is 02:ba:15:00:00:10, so it moves to next step) compare target ip received to probe sent IP, if not continue (the probe sent IP source 0.0.0.0, the grat arp had a target ip 0.0.0.0 soe it moves to the next step) return -1 (ARP probe failed)

(for probe responses, shouldbe be checking target or source, ie wouldn't the arp reply of an 'already-used' IP address have it's IP/MAC in the ARP sender field instead of target field Suspect maybe it should be:

if (memcmp(ifr.ifr_hwaddr.sa_data, hdr.sender_mac, 6) != 0)

and

memcpy(&ip, hdr.sender_ip, 4)

instead

osevan commented 9 months ago

Is this related with

https://github.com/netblue30/firejail/discussions/6102

?

brianvanderburg2 commented 9 months ago

I don't think so. In your log file the network is already established and enters the jail. In mine, it fails at assigning an IP address and does not even enter the sandbox.

Looking through the code and packet dump it makes sense.

My system sends: sender mac: 02:ba:15:00:00:10 sender ip: 0.0.0.0 target mac: 00:00:00:00:00:00 target ip: 192.168.1.30

This other system on the network send a reply: sender mac: f0:d4:15:45:5d:5d sender ip: 192.168.1.122 target mac: 02:ba:15:00:00:10 target ip: 0.0.0.0

The code is checking if the target values of the reply are the same as the sender of the probe, which they are, so returns -1, and arp_random/arp_sequential then test the result, which is not 0, so returns 0 for the "ip" which then causes arp_assign to fail.

image

This seems to confirm it should compare the received frame's sender IP with the probe frame's target IP, and if the same, treat as duplicate. In the code, it compares the received frame's target IP (hdr.target_ip) with the probe frame's sender IP (srcaddr) Here I don't think the MAC even matters, another ARP being received with the same sender IP that we are trying to use would mean a duplicate

It also recommends if receiving an ARP request during probing with a target ip matching the IP we are trying to use and a source mac not equal to ours, to treat as duplicate as another station may be probing to use the IP at the same time as we are

I don't have build tools on my system right now, but I'm thinking this is what the function should look like

int arp_check(const char *dev, uint32_t destaddr) {
    // RFC 5227 - using a source IP address of 0 for probing
    uint32_t srcaddr = 0;

    if (strlen(dev) > IFNAMSIZ) {
        fprintf(stderr, "Error: invalid network device name %s\n", dev);
        exit(1);
    }

    if (arg_debug)
        printf("Trying %d.%d.%d.%d ...\n", PRINT_IP(destaddr));

    // find interface address
    int sock;
    if ((sock = socket(AF_INET, SOCK_RAW, IPPROTO_RAW)) < 0)
        errExit("socket");

    srcaddr = htonl(srcaddr);
    destaddr = htonl(destaddr);

    // Find interface MAC address
    struct ifreq ifr;
    memset(&ifr, 0, sizeof (ifr));
    strncpy(ifr.ifr_name, dev, IFNAMSIZ - 1);
    if (ioctl(sock, SIOCGIFHWADDR, &ifr) < 0)
        errExit("ioctl");
    close(sock);

    // configure layer2 socket address information
    struct sockaddr_ll addr;
    memset(&addr, 0, sizeof(addr));
    if ((addr.sll_ifindex = if_nametoindex(dev)) == 0)
        errExit("if_nametoindex");
    addr.sll_family = AF_PACKET;
    memcpy (addr.sll_addr, ifr.ifr_hwaddr.sa_data, 6);
    addr.sll_halen = ETH_ALEN;

    // build the arp packet header
    ArpHdr hdr;
    memset(&hdr, 0, sizeof(hdr));
    hdr.htype = htons(1);
    hdr.ptype = htons(ETH_P_IP);
    hdr.hlen = 6;
    hdr.plen = 4;
    hdr.opcode = htons(1); //ARPOP_REQUEST
    memcpy(hdr.sender_mac, ifr.ifr_hwaddr.sa_data, 6);
    memcpy(hdr.sender_ip, (uint8_t *)&srcaddr, 4);
    memcpy(hdr.target_ip, (uint8_t *)&destaddr, 4);

    // build ethernet frame
    uint8_t frame[ETH_FRAME_LEN]; // includes eth header, vlan, and crc
    memset(frame, 0, sizeof(frame));
    frame[0] = frame[1] = frame[2] = frame[3] = frame[4] = frame[5] = 0xff;
    memcpy(frame + 6, ifr.ifr_hwaddr.sa_data, 6);
    frame[12] = ETH_P_ARP / 256;
    frame[13] = ETH_P_ARP % 256;
    memcpy (frame + 14, &hdr, sizeof(hdr));

    // open layer2 socket
    if ((sock = socket(PF_PACKET, SOCK_RAW, htons (ETH_P_ALL))) < 0)
        errExit("socket");

    if (sendto (sock, frame, 14 + sizeof(ArpHdr), 0, (struct sockaddr *) &addr, sizeof (addr)) <= 0)
        errExit("send");
    fflush(0);

    // send two probes at 0.5 seconds interva;
    int  cnt = checkcfg(CFG_ARP_PROBES);
    uint8_t framerx[ETH_FRAME_LEN]; // includes eth header, vlan, and crc
    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(sock, &fds);
    int maxfd = sock;
    struct timeval ts;
    gettimeofday(&ts, NULL);
    double timerend = ts.tv_sec + ts.tv_usec / 1000000.0 + 0.5;
    while (1) {
        gettimeofday(&ts, NULL);
        double now = ts.tv_sec + ts.tv_usec / 1000000.0;
        double timeout = timerend - now;
        ts.tv_sec = timeout;
        ts.tv_usec = (timeout - ts.tv_sec) * 1000000;
        int nready = select(maxfd + 1,  &fds, (fd_set *) 0, (fd_set *) 0, &ts);
        if (nready < 0)
            errExit("select");
        else if (nready == 0) { // timeout
            if (--cnt <= 0) {
                close(sock);
                return 0;
            }
            if (sendto (sock, frame, 14 + sizeof(ArpHdr), 0, (struct sockaddr *) &addr, sizeof (addr)) <= 0)
                errExit("send");
            gettimeofday(&ts, NULL);
            timerend = ts.tv_sec + ts.tv_usec / 1000000.0 + 0.5;
            fflush(0);
        }
        else {
            // read the incoming packet
            int len = recvfrom(sock, framerx, ETH_FRAME_LEN, 0, NULL, NULL);
            if (len < 0) {
                perror("recvfrom");
                close(sock);
                return -1;
            }

            // parse the incoming packet
            if ((unsigned int) len < 14 + sizeof(ArpHdr))
                continue;
            if (framerx[12] != (ETH_P_ARP / 256) || framerx[13] != (ETH_P_ARP % 256))
                continue;
            memcpy(&hdr, framerx + 14, sizeof(ArpHdr));
            if (hdr.opcode == htons(1))
                continue;
            if (hdr.opcode == htons(2)) {
                // check my mac and my address

                // Does this even matter at this part?
                //if (memcmp(ifr.ifr_hwaddr.sa_data, hdr.target_mac, 6) != 0)
                //  continue;

                // Here, I think we should be testing the sender IP of the received ARP reply with the dstaddr 
                // we were originally probing for. If equal, then another host send a reply claiming to the be
                // the IP address we want to use
                uint32_t ip;
                //memcpy(&ip, hdr.target_ip, 4);
                memcpy(&ip, hdr.sender_ip, 4);
                //if (ip != srcaddr) {
                if (ip != dstaddr) {
                    continue;
                }
                close(sock);
                return -1;
            }
        }
    }

    __builtin_unreachable();
}