Use TPROXY instead of nfqueue

[glaslos]

We currently use nfqueue to redirect the packet from an arbitrary port to our honeypot by rewriting the destination port on incoming packets and reverting the changes on outgoing packets. This works well but adds complexity and overhead to the system.

Cloudflare has a blog post outlining the approach which is almost precisely what we want.

The blog boils down to:

ip route add local 192.0.2.0/24 dev lo src 127.0.0.1
iptables -t mangle -I PREROUTING -d 192.0.2.0/24 -p tcp -j TPROXY --on-port 5000 --on-ip 127.0.0.1

This does not work. No external traffic will go to 192.0.2.0/24 and therefor the proxy never comes into play.

If I modify the iptables rule to the following:

`iptables -t mangle -I PREROUTING -p tcp ! -s eth0 ! --dport 22 -j TPROXY --on-port 5000 --on-ip 127.0.0.1`

And adding a listener on port 5000 with IP_TRANSPARENT and I get all connections as expected. Initial work can be found here.

The ! -s eth0 is an attempt to exclude all traffic originating from the interface. ! --dport 22 allows all ssh traffic into the server making sure we don't lock ourselves out.

I didn't use their AnyCast approach which I suspect is the problem, as it seems like there is some routing information missing.

The struggle so far is making sure outgoing packets have a route and don't end up in the proxy. Occasionally I want to establish outgoing TCP connections and if I read it correctly, the SYN packets go out as expected but any response gets picked up by the TPROXY rule and is dropped.

I tried adding a route for outgoing connection like this:

echo "10 tproxy" >> /etc/iproute2/rt_tables
ip rule add from 10.x.x.81 table tproxy
ip route add default via 10.x.x.80 dev ens2 table tproxy

And update the iptables rule to:

-A PREROUTING ! -s 10.x.x.81/32 -i ens2 -p tcp -m tcp ! --dport 22 -j TPROXY --on-port 5000 --on-ip 127.0.0.1

Still with the same issue:

11:42:28.172268 IP hostname.46324 > remote.https: Flags [S], seq 2783956497, win 64240, options [mss 1460,sackOK,TS val 1288228251 ecr 0,nop,wscale 7], length 0
11:42:28.173479 IP remote.https > hostname.46324: Flags [S.], seq 573845087, ack 2783956498, win 65160, options [mss 1460,sackOK,TS val 3249746343 ecr 1288227230,nop,wscale 7], length 0
11:42:29.202975 IP remote.https > hostname.46324: Flags [S.], seq 573845087, ack 2783956498, win 65160, options [mss 1460,sackOK,TS val 3249747372 ecr 1288227230,nop,wscale 7], length 0
11:42:29.620887 IP hostname.46334 > remote.https: Flags [S], seq 1774017241, win 64240, options [mss 1460,sackOK,TS val 1288229700 ecr 0,nop,wscale 7], length 0
11:42:29.621686 IP remote.https > hostname.46334: Flags [S.], seq 1078655880, ack 1774017242, win 65160, options [mss 1460,sackOK,TS val 3249747791 ecr 1288229700,nop,wscale 7], length 0

SOLUTION: iptables -t mangle -I PREROUTING -p tcp ! --dport 22 -m state ! --state ESTABLISHED,RELATED -j TPROXY --on-port 5000 --on-ip 127.0.0.1

[glaslos]

@dkumiszhan this might be an interesting thing to try for you. Be aware that the draft PR has a couple of hardcoded stuff in the iptables rules.

[glaslos]

Closed with https://github.com/mushorg/glutton/pull/153