Modes supported - Githubissues

mrbluecoat commented 1 month ago

Out of curiosity, which modes does durdur support? https://man.archlinux.org/man/xdp-loader.8.en#-m,_--mode

(my use case requires SKB mode)

boratanrikulu commented 1 month ago

It uses default one - Generic (SKB) mode;

type XDPOptions struct {
    // Program must be an XDP BPF program.
    Program *ebpf.Program

    // Interface is the interface index to attach program to.
    Interface int

    // Flags is one of XDPAttachFlags (optional).
    //
    // Only one XDP mode should be set, without flag defaults
    // to driver/generic mode (best effort).
    Flags XDPAttachFlags
}

const (
    // XDPGenericMode (SKB) links XDP BPF program for drivers which do
    // not yet support native XDP.
    XDPGenericMode XDPAttachFlags = 1 << (iota + 1)
    // XDPDriverMode links XDP BPF program into the driver’s receive path.
    XDPDriverMode
    // XDPOffloadMode offloads the entire XDP BPF program into hardware.
    XDPOffloadMode
)

mrbluecoat commented 1 month ago

hmm.. any ideas why I'm getting this then?

# durdur attach --interface enp0s6
Error: attach: failed to attach link: create link: invalid argument

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.19/24 metric 100 brd 10.0.0.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fe80::17ff:xxxx:xxxx/64 scope link 
       valid_lft forever preferred_lft forever

# uname -a
Linux mynode 6.5.0-1026-oracle #26~22.04.1-Ubuntu SMP Mon Jun 24 14:22:18 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

boratanrikulu commented 1 month ago

Not sure :thinking: never tested with ARM

How did you build the application?

mrbluecoat commented 1 month ago

Not sure 🤔 never tested with ARM

How did you build the application?

https://github.com/boratanrikulu/durdur/issues/9#issue-2433515127

boratanrikulu commented 1 month ago

Can you check this doc https://trying2adult.com/what-is-xdp-and-how-do-you-use-it-in-linux-amazon-ec2-example/

mrbluecoat commented 1 month ago

Unfortunately, no luck:

# ip link set dev enp0s6 mtu 3818
# ip a | grep enp0s6.*3818
2: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 3818 qdisc mq state UP group default qlen 1000
# durdur attach --interface enp0s6
Error: attach: failed to attach link: create link: invalid argument

mrbluecoat commented 1 month ago

Not sure if it helps, but https://github.com/ahsifer/goxdp is working (even without the MTU modification)

mrbluecoat commented 1 month ago

P.S. If you want to test on ARM, Oracle offers free instances: https://mrbluecoat.blogspot.com/2021/06/oracle-cloud-arm-most-generous-free-tier.html

boratanrikulu commented 1 month ago

interesting

Can you try to change eBPF-go version in go.mod file v0.12.3 and run these commands

go mod tidy
make build
sudo ./build/durdur attach -i enp0s6

module github.com/boratanrikulu/durdur

go 1.21

toolchain go1.22.5

require (
    github.com/cilium/ebpf v0.12.3
    github.com/frankban/quicktest v1.14.5
    github.com/urfave/cli/v2 v2.23.7
    golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2
)

require (
    github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
    github.com/google/go-cmp v0.6.0 // indirect
    github.com/kr/pretty v0.3.1 // indirect
    github.com/kr/text v0.2.0 // indirect
    github.com/rogpeppe/go-internal v1.11.0 // indirect
    github.com/russross/blackfriday/v2 v2.1.0 // indirect
    github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
    golang.org/x/sys v0.20.0 // indirect
)

mrbluecoat commented 1 month ago

Sorry, no luck:

# cat go.mod 
module github.com/boratanrikulu/durdur

go 1.21

toolchain go1.22.5

require (
        github.com/cilium/ebpf v0.12.3
        github.com/frankban/quicktest v1.14.5
        github.com/urfave/cli/v2 v2.23.7
        golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2
)

require (
        github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
        github.com/google/go-cmp v0.6.0 // indirect
        github.com/kr/pretty v0.3.1 // indirect
        github.com/kr/text v0.2.0 // indirect
        github.com/rogpeppe/go-internal v1.11.0 // indirect
        github.com/russross/blackfriday/v2 v2.1.0 // indirect
        github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
        golang.org/x/sys v0.20.0 // indirect
)
# go mod tidy
go: downloading github.com/frankban/quicktest v1.14.5
# make build
go generate ./internal/generated...
Compiled /tmp/durdur/internal/generated/bpf_bpfeb.o
Stripped /tmp/durdur/internal/generated/bpf_bpfeb.o
Wrote /tmp/durdur/internal/generated/bpf_bpfeb.go
Compiled /tmp/durdur/internal/generated/bpf_bpfel.o
Stripped /tmp/durdur/internal/generated/bpf_bpfel.o
Wrote /tmp/durdur/internal/generated/bpf_bpfel.go
CGO_ENABLED=0 go build -o build/durdur ./cmd/durdur
# ./build/durdur attach -i enp0s6
Error: attach: create link: invalid argument

boratanrikulu commented 1 month ago

you have root while running the program, right?

I set up ARM CI here: https://github.com/boratanrikulu/durdur/actions/runs/10126646992/job/28003538415

Linux vm0dcmer 6.5.0-1023-azure #24~22.04.1-Ubuntu SMP Thu Jun 13 03:54:29 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Attach/Detach just works fine. But drop not working. Probably some bugs in eBPF code.

mrbluecoat commented 1 month ago

Turns out MTU 3818 was still too high. https://github.com/cilium/cilium/issues/25453#issuecomment-1600522849 indicated 3498 which worked!

mrbluecoat commented 1 month ago

The default MTU is set to 9001 on the ena driver. Given XDP buffers are linear, they operate on a single page. A driver typically reserves some headroom for XDP as well (e.g. for encapsulation purpose), therefore, the highest possible MTU for XDP would be 3498.

https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/#nodeport-xdp-on-aws

boratanrikulu commented 1 month ago

Aha, nice!

What about Drop. Is it working for you?

sudo ./build/durdur attach -i enp0s6
sudo ./build/durdur drop --src "169.155.49.112"
ping -c 1 -I enp0s6 169.155.49.112 # should show 100% packet loss
sudo ./build/durdur detach

mrbluecoat commented 1 month ago

Yes, appears to be (using the new MTU and the new go.mod). I'll do a bit more testing..

mrbluecoat commented 1 month ago

Awesome, it passed the stress test! <3

# PIDMAX=$(cat /proc/sys/kernel/pid_max)
# 
# echo "fs.file-max = $PIDMAX" >> /etc/sysctl.d/local.conf
# echo "fs.nr_open = $PIDMAX" >> /etc/sysctl.d/local.conf
# sysctl -p /etc/sysctl.d/local.conf
# ulimit -n $PIDMAX
# ulimit -l unlimited
# sed -i "s/# End of file//" /etc/security/limits.conf
# printf "\n* - nofile $PIDMAX\nroot - nofile $PIDMAX\n" >> /etc/security/limits.conf
# printf "\n* - memlock unlimited\nroot - memlock unlimited\n" >> /etc/security/limits.conf
# printf "\nulimit -n $PIDMAX\nulimit -l unlimited\n" >> ~/.bashrc
# 
# durdur attach --interface enp0s6
# 
# wget -q https://github.com/dibdot/DoH-IP-blocklists/raw/master/doh-ipv4.txt
# while read line; do durdur drop --src $(echo "$line" | awk '{print $1}'); done < doh-ipv4.txt
# 
# durdur drop --src $(dig +short example.com)
# 
# durdur list all | jq length
2016
# ping -c1 example.com
PING example.com (93.184.215.14) 56(84) bytes of data.

--- example.com ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
# ping -c1 google.com
PING google.com (74.125.202.113) 56(84) bytes of data.
64 bytes from io-in-f113.1e100.net (74.125.202.113): icmp_seq=1 ttl=61 time=55.8 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 55.816/55.816/55.816/0.000 ms
#

boratanrikulu commented 1 month ago

Nice, but somehow it's not working with arm ci :smile: https://github.com/boratanrikulu/durdur/pull/11

mrbluecoat commented 1 month ago

LOL, I'll reopen so you can figure that one out :smile:

boratanrikulu commented 1 month ago

Looks like working with lo interface but not working with ens3 interface :smile:

mrbluecoat commented 1 month ago

Also, drop works for IP addresses (src) but not DNS:

# durdur list dns
{"example.com":4}
# ping -c1 -I enp0s6 example.com
PING example.com (93.184.215.14) from 10.0.0.19 enp0s6: 56(84) bytes of data.
64 bytes from 93.184.215.14 (93.184.215.14): icmp_seq=1 ttl=58 time=8.68 ms

--- example.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 8.682/8.682/8.682/0.000 ms
# durdur drop --src $(dig +short badexample.com)
# ping -c1 -I enp0s6 badexample.com
PING badexample.com (173.236.193.195) from 10.0.0.19 enp0s6: 56(84) bytes of data.

--- badexample.com ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

mrbluecoat commented 1 month ago

Looks like working with lo interface but not working with ens3 interface 😄

What's the MTU on ens3?

boratanrikulu commented 1 month ago

Also, drop works for IP addresses (src) but not DNS:

Can you test DNS like this, without using DNS cache;

dig @8.8.8.8 bora.sh +short

DNS/DROP only working for DNS record checkings

boratanrikulu commented 1 month ago

What's the MTU on ens3?

1400

mrbluecoat commented 1 month ago

Can you test DNS like this, without using DNS cache;

Oh, I see.

# dig @8.8.8.8 +short example.com
;; communications error to 8.8.8.8#53: timed out

I guess this is only useful if the server has no upstream recursive DNS servers, right? (e.g. exclusively localhost DNS authoritative server)

boratanrikulu commented 1 month ago

❯ sudo ./build/durdur drop --dns "bora.sh"
❯ dig @127.0.0.1 bora.sh +short
;; communications error to 127.0.0.1#53: timed out
❯ sudo ./build/durdur undrop --dns "bora.sh"
❯ dig @127.0.0.1 bora.sh +short
172.67.137.35
104.21.26.157

I guess should be fine with local dns servers too. But DoH is not supported, if you have enabled DoH inside your local DNS server, durdur can't catch it.

mrbluecoat commented 1 month ago

If durdur is running on a proxy and I want to block a domain but not the IP (e.g. for situations where a shared host is using a single IP for hosting multiple domains) is it possible? I see the dig is blocked but curl can still access the website and devices connecting to the proxy server can still access the website.

For example, durdur drop --dns 'example.com' doesn't work but echo '0.0.0.0 example.com' >> /etc/hosts does.

boratanrikulu commented 1 month ago

It should be working with durdur too

❯ sudo ./build/durdur list all
{"127.0.0.8":3,"bora.sh":4}
❯ curl bora.sh
curl: (6) Could not resolve host: bora.sh

Are you using DoH?

boratanrikulu commented 1 month ago

❯ curl https://www.google.com -I 2&>1 | head -n1
HTTP/2 200 
❯ sudo ./build/durdur drop --dns "www.google.com"
❯ curl https://www.google.com -I
curl: (6) Could not resolve host: www.google.com

boratanrikulu commented 1 month ago

Maybe there are some logic issues, not sure

if (ip->protocol == IPPROTO_UDP)
{
    struct udphdr *udp;
    if (data + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr) > data_end)
    {
        return XDP_PASS;
    }

    udp = data + sizeof(struct ethhdr) + sizeof(struct iphdr);
    if (udp->source == bpf_htons(53))
    {
        if (data + sizeof(*eth) + sizeof(*ip) + sizeof(*udp) + sizeof(struct dnshdr) > data_end)
        {
            return XDP_PASS;
        }

        struct dnshdr *dns = data + sizeof(*eth) + sizeof(*ip) + sizeof(*udp);
        if (dns->opcode == 0) // it's a dns query.
        {
            void *query_start = (void *)dns + sizeof(struct dnshdr);

            struct dnsquery query;
            if (!parse_query(data_end, query_start, &query))
            {
                return XDP_PASS;
            }

            long *pkt_count = bpf_map_lookup_elem(&drop_dns, &query.name);
            if (pkt_count)
            {
                printk("[BLOCK] DNS QUERY TO %s", &query.name);
                __sync_fetch_and_add(pkt_count, 1);
                return XDP_DROP;
            }
            printk("[ALLOW] DNS QUERY TO %s", &query.name);
        }
    }
}

boratanrikulu commented 1 month ago

if (udp->source == bpf_htons(53))

Do you use different port for your local DNS server?

mrbluecoat commented 1 month ago

# curl -si https://www.google.com | head -n1
HTTP/2 200 
# durdur drop --dns "www.google.com"
# durdur list dns
{"example.com":0,"www.google.com":0}
# curl -si https://www.google.com | head -n1
HTTP/2 200 
# nslookup www.google.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
www.google.com  canonical name = forcesafesearch.google.com.
Name:   forcesafesearch.google.com
Address: 216.239.38.120
Name:   forcesafesearch.google.com
Address: 2001:4860:4802:32::78

mrbluecoat commented 1 month ago

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

mrbluecoat commented 1 month ago

dig @8.8.8.8 bora.sh +short

As a completely random aside, I see you're developing an observability and monitoring agent at Sematext. Is durdur the spiritual successor to https://github.com/sematext/oxdpus ?

boratanrikulu commented 1 month ago

No it's not. These are different projects. Durdur is not directly related to Sematext

boratanrikulu commented 1 month ago

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

Maybe they have DoH enabled. I'm not sure if we can catch DNS in that case

mrbluecoat commented 1 month ago

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

I figured it out. For IP blocking (src), I have to attach to the WAN NIC but for DNS blocking I have to attach to the tailscale0 virtual interface:

# ip a | grep tailscale
3: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 xdpgeneric/id:4117 qdisc fq_codel state UNKNOWN group default qlen 500
    inet 100.103.191.142/32 scope global tailscale0

Makes sense in hindsight. Can I run two instances of durdur concurrently on different interfaces?

boratanrikulu commented 1 month ago

Right now we don't have support for multiple interfaces. But I guess it would be easy to add that feature.

mrbluecoat commented 1 month ago

That would be awesome. A comma-separated option would be fine for my needs, like: https://github.com/ahsifer/goxdp/blob/main/README.md?plain=1#L75-L78

boratanrikulu / durdur

Modes supported #10