(Destination port updating) Forward HTTP requests to another webserver (port) BPF/XDP

[Fabian-Ser]

I am trying to route my GET requests to another port using XDP.

I created the following code:

 int xdp_program(struct xdp_md *ctx)
    {
        void *data_end = (void *)(long)ctx->data_end;
        void *data = (void *)(long)ctx->data;

        struct ethhdr *eth = data;
        if (eth + 1 > (struct ethhdr *)data_end)
        {
            bpf_printk("Invalid ETHERNET header");
            return XDP_DROP;
        }

        struct iphdr *iph = (data + sizeof(struct ethhdr));
        if (iph + 1 > (struct iphdr *)data_end)
        {
            bpf_printk("Invalid IP header");
            return XDP_DROP;
        }

        if(iph->protocol == IPPROTO_TCP) {
            struct tcphdr *tcph = (data + sizeof(struct ethhdr) + sizeof(struct iphdr));
            if (tcph + 1 > (struct tcphdr *)data_end)
            {
                bpf_printk("Invalid TCP header");
                return XDP_DROP;
            }

            if (tcph->dest == htons(8000))
            {
                if(GetPayload(ctx, eth, iph, tcph) == 1) {
                     tcph->dest = htons(289);
                }    
                return XDP_PASS;
            }
        }
}

I have another webserver running on port 289 which is working correctly when accessed directly, unfortunately, the page isn't showing that webserver. When logging port 289 in XDP, nothing shows up. When using TCPDump on the server, I got a couple of RST and PSH packets. What do I wrong/forget in order to show the webserver running on port 289?

[tohojo]

For the TCP packets to be valid you also need to update the checksum field when you're updating the header. Also, you're doing one side of a NAT translation; however, for NAT to work you need some mechanism to also translate the reply packets back to the original port number, or the sender will get confused...

[Fabian-Ser]

Thanks @tohojo ,

I tried to add an tcp->check(sum). I tried the following:

static inline unsigned short checksum(unsigned short *buf, int bufsz) {
    unsigned long sum = 0;

    while (bufsz > 1) {
        sum += *buf;
        buf++;
        bufsz -= 2;
    }

    if (bufsz == 1) {
        sum += *(unsigned char *)buf;
    }

    sum = (sum & 0xffff) + (sum >> 16);
    sum = (sum & 0xffff) + (sum >> 16);

    return ~sum;
}

Where I call it the following way:

tcph->check = 0;
tcph->check = checksum((unsigned short *)tcph, sizeof(struct tcphdr));

This creates also SYN/ACK packets logging in TCPDump, which is a better result. But there is still a "TCP checksum invalid" notice. Is there a better way to create a checksum (as this one doesn't work properly?)

For the part of replying, I assume I need XDP_TX instead of the XDP_PASS I use. (as the packet is changed)?

Thanks!

[tohojo]

You should be able to do an incremental checksum update instead of computing the whole thing; like in this: https://github.com/xdp-project/xdp-tools/blob/xdp-trafficgen/xdp-trafficgen/xdp-trafficgen.kern.c#L261-L267

As for handling the return packets, no, it's not a question of just returning XDP_TX. When you XDP_PASS the first packet, that will go to the TCP stack, and the stack will produce a reply going out. That packet needs to be rewritten so that it has the port number expected by the other end of the connection. There is no way to do this with XDP (as it doesn't have a TX hook), but you could do it using TC-BPF on the egress interface...