switch does not respond to routed packets with a ttl of 1

[disaster123]

Running a traceroute in standard udp mode or in -I icmp mode results in * on S4248fbl-ON device for the opx hop.

The packets are correctly arriving at the opx device with a ttl of 1:

# tcpdump -vvv -n -i any host 37.228.152.11
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
23:12:59.286636 IP (tos 0x0, ttl 1, id 44183, offset 0, flags [none], proto ICMP (1), length 60)
    37.228.152.11 > 1.1.1.1: ICMP echo request, id 11188, seq 1, length 40
23:12:59.286704 IP (tos 0x0, ttl 1, id 44184, offset 0, flags [none], proto ICMP (1), length 60)
    37.228.152.11 > 1.1.1.1: ICMP echo request, id 11188, seq 2, length 40
23:12:59.286756 IP (tos 0x0, ttl 1, id 44185, offset 0, flags [none], proto ICMP (1), length 60)
    37.228.152.11 > 1.1.1.1: ICMP echo request, id 11188, seq 3, length 40

but the opx device / kernel does not send any icmp response :-(

ping @mikelazar and @gunasekaran-t

Greets, Stefan

[disaster123]

This only happens with routed pkts with ttl=1 - directly pinging the switch works fine and triggers an imcp echo reply.

So the echo reply time too live exceeded does not work.

Is this somehow related to? https://github.com/open-switch/opx-base-model/blob/abdf66f813b48a3c8e7682361cdacccd0271866d/yang-models/os-icmp-config.yang

[disaster123]

A tcpdump -vvv -e -n -i any icmp never shows any packet containing time to live exceeded - is this broken on OPX?

[disaster123]

got it fixed by enabling forwarding for all interfaces. Is this a bug in OPX? all interfaces got installed with net.ipv4.conf.e101-047-0.forwarding = 0 even net.ipv4.conf.default.forwarding is set

[gunasekaran-t]

@disaster123 I've verified in S4200FBL platform with latest packages and I could see the TTL exceeded message is being sent out and traceroute gets completed.

Also forwarding is enabled in all the interfaces by default . Please share the steps if you can identify what could make the forwarding to be disabled.

~# traceroute -I 2.1.1.2  
traceroute to 2.1.1.2 (2.1.1.2), 30 hops max, 60 byte packets
 1  1.1.1.1 (1.1.1.1)  1.383 ms  1.530 ms  1.668 ms
 2  2.1.1.2 (2.1.1.2)  2.367 ms  2.926 ms  1.528 ms
~#

# tcpdump In S4200

21:22:25.504519 IP (tos 0x0, ttl 1, id 54774, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 1, length 40
21:22:25.504629 IP (tos 0xc0, ttl 64, id 64799, offset 0, flags [none], proto ICMP (1), length 88)
    1.1.1.1 > 1.1.1.2: ICMP time exceeded in-transit, length 68
        IP (tos 0x0, ttl 1, id 54774, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 1, length 40
21:22:25.504759 IP (tos 0x0, ttl 1, id 54775, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 2, length 40
21:22:25.504812 IP (tos 0xc0, ttl 64, id 64800, offset 0, flags [none], proto ICMP (1), length 88)
    1.1.1.1 > 1.1.1.2: ICMP time exceeded in-transit, length 68
        IP (tos 0x0, ttl 1, id 54775, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 2, length 40
21:22:25.504952 IP (tos 0x0, ttl 1, id 54776, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 3, length 40
21:22:25.505033 IP (tos 0xc0, ttl 64, id 64801, offset 0, flags [none], proto ICMP (1), length 88)
    1.1.1.1 > 1.1.1.2: ICMP time exceeded in-transit, length 68 
        IP (tos 0x0, ttl 1, id 54776, offset 0, flags [none], proto ICMP (1), length 60)
    1.1.1.2 > 2.1.1.2: ICMP echo request, id 6046, seq 3, length 40

root@OPX:/home/admin# sysctl net.ipv4.conf | grep '\.forwarding' | grep 041
net.ipv4.conf.e101-041-1.forwarding = 1
net.ipv4.conf.e101-041-2.forwarding = 1
net.ipv4.conf.e101-041-3.forwarding = 1
net.ipv4.conf.e101-041-4.forwarding = 1
root@OPX:/home/admin#
root@OPX:/home/admin# sysctl net.ipv4.conf | grep '\.forwarding' | grep bond10
net.ipv4.conf.bond10.forwarding = 1
root@OPX:/home/admin# sysctl net.ipv4.conf | grep \.forwarding' | grep br10
net.ipv4.conf.br10.forwarding = 1
root@OPX:/home/admin#

root@OPX:/home/admin# sysctl net.ipv4.conf | grep '\.forwarding' | wc
     65     195    2536
root@OPX:/home/admin# sysctl net.ipv4.conf | grep '\.forwarding = 1'  | wc
     65     195    2536
root@OPX:/home/admin#