ICMP proxy: Why we need to rewrite when `icmp_hdr.type() == icmp_header::time_exceeded`?

leiless commented 4 years ago

Hi, @yuchting, I recently working on a small and simple project leiless/icmp-redir, which essentially is an ICMP proxy as what you already did.

What I have done:

Setup raw socket
Listening for ICMP packet
When received an ICMP packet, rewrite IP header's saddr to INADDR_ANY, daddr to ICMP proxy server address.
Send out rewrote ICMP packet
When ICMP_ECHOREPLY received, rewrote it to construct the proper ICMP reply to DHCP client.
Whole round trip of ICMP proxy done(hopefully)

After I wrote the demo program, I found that mtr and traceroute seems broken, as you have shown in Can we proxy ICMP message (To transfer ping), which your solution worked as expected.

While mine certainly has some conceptual misunderstanding.

My solution loosely can be described as the following flow chart: ICMP_redir_v3

The LAN IP is the DHCP client.

Also, I've checked your code, it seems that you've done some special treatments upon traceroute:

    } else if (icmp_hdr.type() == icmp_header::time_exceeded) { // for traceroute
        ipv4_header orig_ipv4_hdr;
        icmp_header orig_icmp_hdr;
        string orig_body;
        std::istringstream orig_is(body);
        if (read_icmp(orig_is, body.length(), orig_ipv4_hdr, orig_icmp_hdr, orig_body)) {
            auto hash = orig_ipv4_hdr.destination_address().to_string() +
                        to_string((int)icmp_header::echo_request) +
                        to_string(orig_icmp_hdr.identifier()) +
                        to_string(orig_icmp_hdr.sequence_number());

            icmp_sent_data = find_icmp_sent_data(hash, true);
            if (icmp_sent_data) {
                orig_ipv4_hdr.source_address(icmp_sent_data->source);
                orig_ipv4_hdr.assign_header_checksum();

                std::ostringstream os;
                os << orig_ipv4_hdr << orig_icmp_hdr;

                body = os.str();

                icmp_hdr.assign_checksum(body);
                ipv4_hdr.total_length((unsigned short)(ipv4_hdr.header_length() +
                                                       icmp_header::HEADER_LENGTH + body.length()));
            }
        }
    }

https://github.com/Trojan-Plus-Group/trojan-plus/blob/dev/src/core/icmpd.cpp#L157

I don't quite understand how you solve the problem with the mtr and/or traceroute.

Could you help me to with this? Sorry, I'm a newbie in the network programming realm.

leiless commented 4 years ago

Demo program sample output for ping, run in gateway side(software router)

$ sudo ./icmp-redir client -a 1.1.1.1
Build timestamp: ???
     Build user: ???
    HEAD commit: ???

# Received ICMP_ECHO
IP proto: 1 ver: 4 id: 0xb120 length: 20 tos: 0 tot_len: 84 frag_off: 64 ttl: 63 check: 0x90f8
192.168.100.100 -> 39.156.69.79
ICMP length: 64 header: 8 data: 56 type: 8 code: 0 id: 80 seq: 1 checksum: 0xa2b1
Raw packet hexdump:
00000000  45 00 00 54 b1 20 40 00 3f 01 f8 90 c0 a8 64 64  |E..T. @.?.....dd|
00000010  27 9c 45 4f 08 00 b1 a2 00 50 00 01 80 03 1c 5f  |'.EO.....P....._|
00000020  00 00 00 00 e6 d6 04 00 00 00 00 00 10 11 12 13  |................|
00000030  14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23  |............ !"#|
00000040  24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33  |$%&'()*+,-./0123|
00000050  34 35 36 37                                      |4567|
00000060
---- Rewrote ICMP packet ----
IP proto: 1 ver: 4 id: 0xb120 length: 20 tos: 0 tot_len: 106 frag_off: 64 ttl: 63 check: 0x7188
0.0.0.0 -> 1.1.1.1
ICMP length: 86 header: 8 data: 78 type: 8 code: 0 id: 80 seq: 1 checksum: 0x2ab2
Raw packet hexdump:
00000000  45 00 00 6a b1 20 40 00 3f 01 88 71 00 00 00 00  |E..j. @.?..q....|
00000010  01 01 01 01 08 00 b2 2a 00 50 00 01 80 03 1c 5f  |.......*.P....._|
00000020  00 00 00 00 e6 d6 04 00 00 00 00 00 10 11 12 13  |................|
00000030  14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23  |............ !"#|
00000040  24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33  |$%&'()*+,-./0123|
00000050  34 35 36 37 27 9c 45 4f 5b 30 31 32 33 34 35 36  |4567'.EO[0123456|
00000060  37 38 39 61 62 63 64 65 66 5d                    |789abcdef]|
00000070
106 bytes sent out to raw socket

# Received ICMP_ECHOREPLY
IP proto: 1 ver: 4 id: 0x98a0 length: 20 tos: 0 tot_len: 106 frag_off: 0 ttl: 54 check: 0x4728
1.1.1.1 -> 192.168.1.2
ICMP length: 86 header: 8 data: 78 type: 0 code: 0 id: 80 seq: 1 checksum: 0x2aba
Raw packet hexdump:
00000000  45 00 00 6a 98 a0 00 00 36 01 28 47 01 01 01 01  |E..j....6.(G....|
00000010  c0 a8 01 02 00 00 ba 2a 00 50 00 01 80 03 1c 5f  |.......*.P....._|
00000020  00 00 00 00 e6 d6 04 00 00 00 00 00 10 11 12 13  |................|
00000030  14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23  |............ !"#|
00000040  24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33  |$%&'()*+,-./0123|
00000050  34 35 36 37 27 9c 45 4f 5b 30 31 32 33 34 35 36  |4567'.EO[0123456|
00000060  37 38 39 61 62 63 64 65 66 5d                    |789abcdef]|
00000070
---- Rewrote ICMP packet ----
IP proto: 1 ver: 4 id: 0x98a0 length: 20 tos: 0 tot_len: 84 frag_off: 0 ttl: 54 check: 0x115a
39.156.69.79 -> 192.168.100.100
ICMP length: 64 header: 8 data: 56 type: 0 code: 0 id: 80 seq: 1 checksum: 0xa2b9
Raw packet hexdump:
00000000  45 00 00 54 98 a0 00 00 36 01 5a 11 27 9c 45 4f  |E..T....6.Z.'.EO|
00000010  c0 a8 64 64 00 00 b9 a2 00 50 00 01 80 03 1c 5f  |..dd.....P....._|
00000020  00 00 00 00 e6 d6 04 00 00 00 00 00 10 11 12 13  |................|
00000030  14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23  |............ !"#|
00000040  24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33  |$%&'()*+,-./0123|
00000050  34 35 36 37                                      |4567|
00000060
84 bytes sent out to raw socket

DHCP client ping sample output:

$ ping baidu.com -c1
PING baidu.com (39.156.69.79) 56(84) bytes of data.
64 bytes from 39.156.69.79 (39.156.69.79): icmp_seq=1 ttl=53 time=210 ms

--- baidu.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 210.417/210.417/210.417/0.000 ms

leiless commented 4 years ago

mtr is yet broken:

The RT-AC68U-AF18 is the AP-router, network toplogy:

+--------------|-----------------|----------------------------------+
| DHCP client <=> RT-AC68U-AF18 <=> Software router(run icmp proxy) |
+--------------|-----------------|----------------------------------+

leiless commented 4 years ago

It seems that proxy ICMP message(transfer ping) is implemented in NAT mode, which means the whole packet is forwarded(redirected) to the proxy server?

In my implementation, I simply handwrite the redirect(forward) logic in user space, yet your implementation using NAT mode, which means packets are handled in kernel space, so mtr, traceroute works as expected?

yuchting commented 4 years ago

First of all, Linux's traceroute uses UDP+TTL to test route instead of ICMP+TTL (such as Windows' tracert or mtr ) so that trojan plus' way cannot work for it.

And base RFC document each public router must decrease 1 for TTL of IP packet so by mtr or windows's tracert method, their TTL of ICMP's IP header will be set in different value from 1 to 30, when a middle point receive a IP packet and find the TTL is 1, it won't do NAT and send a ICMP packet with "icmp_header::time_exceeded" to source IP.

That's why you see the code:

} else if (icmp_hdr.type() == icmp_header::time_exceeded) { // for traceroute

the key point is that source IP is my trojan plus server's IP, so I need to capture it and change it's source IP to local IP which was stored by trojan plus server and then transfer it to trojan plus client.

After trojan plus client receive this packet, it will send it to local client.

local client (mtr or windows' tracert) will receive a ICMP with middle public router's IP, right specific ID, right checksum value, It will display right information to user.

Basic knowledge

mtr will use specific ID of ICMP to know which packet I should display to user: TTL / ID 1 / AAA 2 / BBB 3 / CCC ... 30 / XXX

when it receive a ICMP with CCC , it will show the source IP (middle public router) in 3rd line, it's the 3rd hope.

And you need to recalculate a right checksum value after you changing ICMP packet otherwise kernel will drop this packet directly rather than pass it to user application.

leiless commented 4 years ago

OK, thx! I'll try later.

Trojan-Plus-Group / trojan-plus

ICMP proxy: Why we need to rewrite when `icmp_hdr.type() == icmp_header::time_exceeded`? #9

Basic knowledge