insomniacslk / dublin-traceroute

Dublin Traceroute is a NAT-aware multipath tracerouting tool
https://dublin-traceroute.net
BSD 2-Clause "Simplified" License
255 stars 57 forks source link

Not getting the trace even though "modern traceroute" is getting the trace fine. #112

Open ghost opened 2 years ago

ghost commented 2 years ago

So, I have ECMP implemented on my machine is the way that gives different traces for different source ports:

traceroute  --version
Modern traceroute for Linux, version 2.1.0
Copyright (c) 2016  Dmitry Butskoy,   License: GPL v2 or any later
sysctl -w net.ipv4.fib_multipath_hash_fields=0x10 # :: use source port
sysctl -w net.ipv4.fib_multipath_hash_policy=0x3  # :: actually use custom multipath fields
sysctl -w net.ipv4.fib_multipath_use_neigh=1 # :: no dead neighbours
ip route add 8.8.8.8 nexthop via 10.0.2.1 nexthop via 192.168.2.1

traceroute -M udp -p 80 -U  8.8.8.8 --sport=33074
traceroute -M udp -p 80 -U  8.8.8.8 --sport=33075

I am giving -M udp in order to match dublin-traceroute's probe algorithm.

I am running dublin-traceroute like this:

dublin-traceroute -T --npaths=2 -i --sport=33174 -N --delay=1000 46.23.90.181  > trace.json && ../scripts/to_graphviz.py trace.json &&  feh trace.json.png

The above command fails, with both tracer ending in a *. The aforementioned modern traceroute works as expected, with sports 33074 and 33075 giving different traces.

insomniacslk commented 2 years ago

hey @lockywolf , thanks for the report! I cannot reproduce your issue from my network. Can you share an anonymized trace.json so we can debug this more easily?

Also there's a small typo in the command line, the -T argument requires a number or the invocation would fail. I assume this is just a copy-paste issue?

ghost commented 2 years ago
{
        "flows" : 
        {
                "35001" : 
                [
                        {
                                "flowhash" : 33617,
                                "is_last" : true,
                                "name" : "",
                                "received" : null,
                                "rtt_usec" : null,
                                "sent" : 
                                {
                                        "ip" : 
                                        {
                                                "dst" : "my.ip",
                                                "src" : "192.168.2.12",
                                                "ttl" : 1
                                        },
                                        "timestamp" : "1642491053.523497",
                                        "udp" : 
                                        {
                                                "dport" : 15001,
                                                "sport" : 35001
                                        }
                                }
                        }
                ],
                "35002" : 
                [
                        {
                                "flowhash" : 33618,
                                "is_last" : true,
                                "name" : "",
                                "received" : null,
                                "rtt_usec" : null,
                                "sent" : 
                                {
                                        "ip" : 
                                        {
                                                "dst" : "my.ip",
                                                "src" : "192.168.2.12",
                                                "ttl" : 1
                                        },
                                        "timestamp" : "1642491056.551146",
                                        "udp" : 
                                        {
                                                "dport" : 15001,
                                                "sport" : 35002
                                        }
                                }
                        }
                ]
        }
}

To be honest, this json is valid :(. At least, jq refuses to query it.

Any way, the at least the source ips are wrong. With identical settings, sport, dport, UDP, modern traceroute chooses two different source ip addresses.

insomniacslk commented 2 years ago

chooses two different source ip addresses.

Aah this smells like an interface selection issue. Can you check if ip route get 46.23.90.181 gives you the same interface as the the IP address that is used by dublin-traceroute?

ghost commented 2 years ago

No, it doesn't.

And it only returns one interface (the other one).

Whereas traceroute itself would choose the interface depending on the ECMP hashing function; different port -- different interface.

josephcolton commented 2 years ago

I am finding a similar issue. When trying to run dublin-traceroute (Dublin Traceroute 0.4.2) to baidu.com using "dublin-traceroute 39.156.66.10", I get most of the way there, then the traceroute stops prematurely. I get all of the way there using scamper with icmp-paris, udp-paris, and icmp. Using normal traceroute (Modern traceroute for Linux, version 2.1.0), I get about as much as dublin-traceroute, then I get stars for the rest.

LeStegii commented 3 months ago

Sorry for necroing, but I experience similar issues. One example is 35.185.71.154 (belonging to google). I couldn't get paris-traceroute to run on my machines, but with modern traceroute (using tcp) the target is reached in no time and without problems.

Starting dublin-traceroute (probe type: IPv4/UDP)
Traceroute from 0.0.0.0:12345 to 35.185.71.154:33434~33443 (probing 10 paths, min TTL is 1, max TTL is 30, delay is 10 ms)
== Flow ID 33434 ==
1    X (gateway.invalid), IP ID: 27906 RTT 5.28 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
2    X (X), IP ID: 1890 RTT 5.12 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
3    X (X), IP ID: 28342 RTT 5.36 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
4    X (X), IP ID: 38380 RTT 5.45 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
5    X (X), IP ID: 0 RTT 5.17 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
6    X (X), IP ID: 0 RTT 5.22 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
7    X (X), IP ID: 53554 RTT 5.6 ms  ICMP (type=11, code=0) 'TTL expired in transit', MPLS(label=19, experimental=0, bottom_of_stack=1, ttl=1), NAT ID: 0, flow hash: 49033
8    82.195.80.45 (te-1-2-0-0.core1.fra1.ix.f.man-da.net), IP ID: 0 RTT 10.183 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
9    82.195.78.134 (gw1.google.pni.man-da.net), IP ID: 40745 RTT 5.83 ms  ICMP (type=11, code=0) 'TTL expired in transit', NAT ID: 0, flow hash: 49033
10    *

I used -n 10 in this case and all of them failed to reach the target.

traceroute to 35.185.71.154 (35.185.71.154), 30 hops max, 52 byte packets
 1  gateway.invalid (X)  0.221 ms  0.193 ms  0.189 ms
 2  X (X)  0.550 ms  0.547 ms  0.543 ms
 3  X (X)  0.695 ms  0.687 ms  0.610 ms
 4  X (X)  0.687 ms X (X)  0.770 ms  0.699 ms
 5  X (X)  0.728 ms  0.743 ms  0.754 ms
 6  X (X)  0.599 ms  0.487 ms  0.545 ms
 7  X (X)  1.087 ms X (X)  1.165 ms X (X)  0.959 ms
 8  X (X)  1.349 ms X (X)  1.347 ms  1.323 ms
 9  gw1.google.pni.man-da.net (82.195.78.134)  3.810 ms  3.448 ms  3.435 ms
10  154.71.185.35.bc.googleusercontent.com (35.185.71.154)  99.911 ms  100.426 ms  100.416 ms

When using modern traceroute without TCP (should therefore default to UDP) I get similar results as dublin with it stopping at gw1.google.xxx or even before that. So it could just be that this is caused by UDP and not dublin directly (in this case this would probably be related to #42)

If I can provide other information that might be useful, let me know.

insomniacslk commented 3 months ago

hey @LeStegii , the issue seems indeed with UDP rather than with the tool. The TCP probe unfortunately is not implemented, neither in the C++ nor in the Go implementation (see the Feature Matrix). I am open to pull requests, but I don't plan to actively work on it in the foreseeable future because I'm busy with other projects