Question: smoltcp-based client <--> server communication (with `2 tap interfaces`)

[nihalpasham]

Networking noob here - Wasn't sure if this was the right place for this question. So, please let me know if this question needs to be directed some place else.

I'm trying to get a smoltcp-based client to talk to a smoltcp-based server but cant seem to get them to talk to each other.

Tried a couple of things so far,

• Using 2 Tap Interfaces:
  • My setup has 2 tap interfaces (tap0 and tap1). One for the client and the other for the server. The client sends raw ipv6 packets to the server and vice-versa. Both interfaces use the same set of instructions (as laid out in the usage examples)
  • The client is assigned fdaa::1 and the server is assigned fdaa::2.
  • However, my client can't seem to find the server.
• Using a Veth pair:
  • My understanding is that the examples don't support veth ifaces, as it results in an OS error - Invalid argument, when calling TapInterface ::new()

Any help here would be appreciated! :)

Client logs:

(base) root@DESKTOP-M9O6B7J:/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2# cargo run --example hip_initiator
    Finished dev [unoptimized + debuginfo] target(s) in 6.33s
     Running `target/debug/examples/hip_initiator`
[0.0s] (socket::set): [0]: adding
[0.0s] (rustdhipv2::daemon::hipd): HIP_STATE is: Some(Unassociated)
[0.0s] (rustdhipv2::daemon::hipd): Starting HIP BEX
[0.0s] (rustdhipv2::daemon::hipd): Sending I1 packet
[0.0s] (socket::raw): #0:IPv6:0x8b: buffer to send 96 octets
[0.0s] (rustdhipv2::storage::HIPState): (key, value) pair inserted
[0.0s] (iface::ethernet): non-unicast source address
[0.0s] (iface::ethernet): cannot process ingress packet: malformed packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=4e-c3-e8-b3-84-36 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=:: dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
.....
[0.0s] (hip_initiator): poll error: malformed packet
[0.0s] (iface::ethernet): non-unicast source address
[0.0s] (iface::ethernet): cannot process ingress packet: malformed packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=4e-c3-e8-b3-84-36 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=:: dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: malformed packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=4e-c3-e8-b3-84-36 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=4e-c3-e8-b3-84-36 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdaa::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fdaa::2 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdaa::2 missing, silencing until t+3.000s

[whitequark]

• My setup has 2 tap interfaces (tap0 and tap1). One for the client and the other for the server. The client sends raw ipv6 packets to the server and vice-versa. Both interfaces use the same set of instructions (as laid out in the usage examples)

Try setting up your host machine to forward packets between those interfaces. I don't quite recall how exactly to do that; I think the easiest way might be to add them to a bridge, but you could try setting up IP forwarding too.

[nihalpasham]

So far, I've tried linking both tap(s) to a bridge (named br0) with brctl (a tool I came across after some googling) .

brctl addbr br0 ip link set br0 up brctl addif br0 tap0 brctl addif br0 tap1

But haven't added the host interface (i.e. eth0) to the bridge.

To clarify, the suggestion is to add the host interface (i.e. on my machine that's eth0) to the bridge, which should allow packets from tap0 to be routed to tap1 i.e. ( tap0 <--> eth0 <--> tap1) .

[Dirbaio]

This looks like #419 . Can you try with latest master?

(Adding the taps to a bridge should be enough unless you want smoltcp to talk with other machines in your LAN or the outside Internet)

[nihalpasham]

Tried master branch. I get the same result but noticed that the server (or responder) gets an IP --> MAC mapping for the client.

(base) root@DESKTOP-M9O6B7J:/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2# cargo run --example hip_responder
    Finished dev [unoptimized + debuginfo] target(s) in 5.98s
     Running `target/debug/examples/hip_responder`
[0.0s] (socket::set): [0]: adding
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::215:5dff:fe2a:308d dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_responder): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=f2-2b-7a-98-ad-6d dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_responder): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=f2-2b-7a-98-ad-6d dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_responder): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::215:5dff:fe2a:308d dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_responder): poll error: unrecognized packet
[0.0s] (iface::neighbor): filled fdaa::1 => 02-00-00-00-00-02 (was empty)

whereas the client (or initiator) doesn't seem to acquire a similar mapping for the server and I'm not sure why. Client and server impl(s) aren't all that different.

I'm assuming the client's neighbor solicitation message is received but the server's response for some reason does not make it back.

(base) root@DESKTOP-M9O6B7J:/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2# cargo run --example hip_initiator
   Compiling rustdhipv2 v0.1.0 (/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2)
    Finished dev [unoptimized + debuginfo] target(s) in 1m 49s
     Running `target/debug/examples/hip_initiator`
[0.0s] (socket::set): [0]: adding
[0.0s] (rustdhipv2::daemon::hipd): HIP_STATE is: Some(Unassociated)
[0.0s] (rustdhipv2::daemon::hipd): Starting HIP BEX
[0.0s] (rustdhipv2::daemon::hipd): Sending I1 packet
[0.0s] (socket::raw): #0:IPv6:0x8b: buffer to send 96 octets
[0.0s] (rustdhipv2::storage::HIPState): (key, value) pair inserted
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::215:5dff:fe2a:308d dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fe80::100 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdbb::2 missing, silencing until t+3.000s
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::100 dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (iface::ethernet): cannot process ingress packet: unrecognized packet
[0.0s] (iface::ethernet): packet dump follows:
EthernetII src=12-95-6a-37-a4-41 dst=33-33-00-00-00-16 type=IPv6
\ IPv6 src=fe80::215:5dff:fe2a:308d dst=ff02::16 nxt_hdr=Hop-by-Hop hop_limit=1
[0.0s] (hip_initiator): poll error: unrecognized packet
[0.0s] (socket::meta): #0: neighbor fdbb::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fe80::100 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdbb::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdbb::2 silence timer expired, rediscovering
[0.0s] (socket::raw): #0:IPv6:0x8b: sending 96 octets
[0.0s] (iface::ethernet): address fe80::100 not in neighbor cache, sending Neighbor Solicitation
[0.0s] (socket::meta): #0: neighbor fdbb::2 missing, silencing until t+3.000s
[0.0s] (socket::meta): #0: neighbor fdbb::2 silence timer expired, rediscovering

[nihalpasham]

In case, this is needed, a snapshot of my current setup (hardware: WSL2 machine running ubuntu)

(base) root@DESKTOP-M9O6B7J:/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
....
....
5: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:15:5d:2a:30:8d brd ff:ff:ff:ff:ff:ff
20: tap1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br0 state DOWN mode DEFAULT group default qlen 1000
    link/ether f2:2b:7a:98:ad:6d brd ff:ff:ff:ff:ff:ff
21: tap0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 12:95:6a:37:a4:41 brd ff:ff:ff:ff:ff:ff
22: br0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 12:95:6a:37:a4:41 brd ff:ff:ff:ff:ff:ff

(base) root@DESKTOP-M9O6B7J:/mnt/c/Users/Nil/devspace/rust/projects/rusty_hipv2# brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.12956a37a441       no              tap0
                                                        tap1

[nihalpasham]

Update: - in case, anyone happens to come across a similar issue, the following may prove useful.

So, WSL 2 (or windows subsystem for linux) distributions have limited IPv6 networking support. Microsoft is working on the issue. More specifically, you cannot reach IPv6-only addresses from a WSL 2 instance. Although, they haven't explicitly stated whether this applies to tap interfaces, I'm presuming its does.

Anyway, I switched to using IPv4 addresses (after some refactoring as my initial target was IPv6-only). It works as expected now.

Steps to build a linux bridge and wire-up 2 tap interfaces:

• brctl addbr br0
• ip link set br0 up
• brctl addif br0 tap0 tap1

[Dirbaio]

Not sure if WSL is the issue, the traffic here shouldn't leave the Linux kernel.

From a quick look at your code, one possible issue I see is that the two IPs you chose are in different subnets:

    let initiator_addr = IpAddress::from_str("fdaa:0:0:0:0:0:0:1").expect("invalid address format");
    let responder_addr = IpAddress::from_str("fdbb:0:0:0:0:0:0:2").expect("invalid address format");

Try picking 2 addrs in the same subnet. fdaa::1/64 and fdaa::2/64, for example

[nihalpasham]

Using ip addresses within the same subnet was actually one of my first attempts.

Also tried a number of other permutations. But at some point, I noticed a regular ping6 from tap0 to tap1 didn't work either, whereas ipv4 pings worked fine..

[pothos]

I can recommend to use macvtap devices¹ in bridge mode, they are easy to set up and you can also use them to speak to the Internet (only the kernel network stack needs an additional macvtap in bridge mode to be able to talk to smoltcp).

You can also use the veth pair if you setup smoltcp to use a RAW socket on each (and the same IP address) which, however, also requires an iptables rule to prevent the kernel sending RST TCP packets for ports it does not know.

¹ https://github.com/ANLAB-KAIST/usnet_devices (but I need to review the PR first)

[nihalpasham]

I'll try the macvtap option and report back (sometime later, as I'm well into my current project which has switched to using an ipv4 setup).

Using a Veth pair: My understanding is that the examples don't support veth ifaces, as it results in an OS error - Invalid argument, when calling TapInterface ::new()

My original idea was to use a veth pair as I'm using RAW sockets (with and without same IP addresses) but kept running into the above error.

Note:- I couldn't get iptables working at all in WSL2.

[pothos]

My understanding is that the examples don't support veth ifaces, as it results in an OS error - Invalid argument, when calling TapInterface ::new()

Instead of trying to use the veth interface like a tap interface you would use a RAW socket on top of the veth interface (RawSocket::new("vethX")).

[nihalpasham]

will try this.