examples/gnrc_border_router not routing between 6lo clients and Internet

benpicco commented 4 years ago

Description

I have seen a similar issue on esp* as border router with a direct WiFi up-link, but sniffing WiFi packets is a bit of a pain, so I tried to reproduce this using Ethernet (CDC-ECM).

The issue I see is that the border router can reach hosts outside the WPAN while it's clients cant. There is some non-determinism involved as sometimes this works, but often it does not.

Steps to reproduce the issue

This requires an upstream Router with DHCPv6 Prefix Delegation enabled. On the common Fritz!Box, this can be enabled in Network -> Network Settings -> IPv6 Addresses:

In the following, we will request a prefix for delegation from the upstream router, then delegate that prefix to the border router so it can be distributed to the 6LoWPAN clients.

But first, let's flash the border router:

make BOARD=samr21-xpro UPLINK=cdc-ecm USE_DHCPV6=1

(This requres a second micro USB cabled plugged to the TARGET USB port of the samr21-xpro)

We should now have a usb network interface, udev gives it a unique name based on the USB topology.

ifconfig output

``` eno1: flags=4163 mtu 1500 inet 192.168.42.69 netmask 255.255.255.0 broadcast 192.168.42.255 inet6 fe80::13ae:a2d4:3402:38fe prefixlen 64 scopeid 0x20 inet6 2001:16b8:45ca:b600:c732:1a76:eedb:8035 prefixlen 64 scopeid 0x0 ether 14:da:e9:4c:6b:2e txqueuelen 1000 (Ethernet) RX packets 1355481 bytes 1502693074 (1.5 GB) RX errors 0 dropped 61432 overruns 0 frame 0 TX packets 552454 bytes 72575305 (72.5 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 20 memory 0xf7f00000-f7f20000 enx0e504ad43426: flags=4163 mtu 1500 inet 169.254.1.2 netmask 255.255.0.0 broadcast 169.254.255.255 inet6 fe80::8520:5adc:6f1c:36f4 prefixlen 64 scopeid 0x20 ether 0e:50:4a:d4:34:26 txqueuelen 1000 (Ethernet) RX packets 13 bytes 1140 (1.1 KB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 198 bytes 38194 (38.1 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ```

I used wide-dhcpv6-client as this will automatically configure the interface.

dhcp6c.conf

``` interface eno1 { send ia-pd 1; }; id-assoc pd 1 { prefix ::/60 infinity; prefix-interface enx0e504ad43426 { sla-id 2; ifid 1; sla-len 4; }; }; ```

sudo dhcp6c -c dhcp6c.conf -D -f eno1

output

``` Jul/31/2020 22:37:40: dhcp6_reset_timer: reset a timer on eno1, state=INIT, timeo=0, retrans=424 Jul/31/2020 22:37:40: client6_send: a new XID (27a739) is generated Jul/31/2020 22:37:40: copy_option: set client ID (len 14) Jul/31/2020 22:37:40: copy_option: set elapsed time (len 2) Jul/31/2020 22:37:40: copyout_option: set IA_PD prefix Jul/31/2020 22:37:40: copyout_option: set IA_PD Jul/31/2020 22:37:40: client6_send: send solicit to ff02::1:2%eno1 Jul/31/2020 22:37:40: dhcp6_reset_timer: reset a timer on eno1, state=SOLICIT, timeo=0, retrans=1061 Jul/31/2020 22:37:40: client6_recv: receive advertise from fe80::de39:6fff:fe6a:6980%eno1 on eno1 Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option client ID, len 14 Jul/31/2020 22:37:40: DUID: 00:01:00:01:26:b7:24:ff:14:da:e9:4c:6b:2e Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option server ID, len 10 Jul/31/2020 22:37:40: DUID: 00:03:00:01:dc:39:6f:6a:69:80 Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option preference, len 1 Jul/31/2020 22:37:40: preference: 0 Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option DNS, len 16 Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option opt_86, len 16 Jul/31/2020 22:37:40: dhcp6_get_options: unknown or unexpected DHCP6 option opt_86, len 16 Jul/31/2020 22:37:40: dhcp6_get_options: get DHCP option IA_PD, len 41 Jul/31/2020 22:37:40: IA_PD: ID=1, T1=1800, T2=2880 Jul/31/2020 22:37:40: copyin_option: get DHCP option IA_PD prefix, len 25 Jul/31/2020 22:37:40: copyin_option: IA_PD prefix: 2001:16b8:45ca:b6c0::/60 pltime=3600 vltime=7200 Jul/31/2020 22:37:40: client6_recvadvert: server ID: 00:03:00:01:dc:39:6f:6a:69:80, pref=0 Jul/31/2020 22:37:40: client6_recvadvert: reset timer for eno1 to 0.996041 Jul/31/2020 22:37:41: select_server: picked a server (ID: 00:03:00:01:dc:39:6f:6a:69:80) Jul/31/2020 22:37:41: client6_send: a new XID (e7c212) is generated Jul/31/2020 22:37:41: copy_option: set client ID (len 14) Jul/31/2020 22:37:41: copy_option: set server ID (len 10) Jul/31/2020 22:37:41: copy_option: set elapsed time (len 2) Jul/31/2020 22:37:41: copyout_option: set IA_PD prefix Jul/31/2020 22:37:41: copyout_option: set IA_PD Jul/31/2020 22:37:41: client6_send: send request to ff02::1:2%eno1 Jul/31/2020 22:37:41: dhcp6_reset_timer: reset a timer on eno1, state=REQUEST, timeo=0, retrans=1074 Jul/31/2020 22:37:41: client6_recv: receive reply from fe80::de39:6fff:fe6a:6980%eno1 on eno1 Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option client ID, len 14 Jul/31/2020 22:37:41: DUID: 00:01:00:01:26:b7:24:ff:14:da:e9:4c:6b:2e Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option server ID, len 10 Jul/31/2020 22:37:41: DUID: 00:03:00:01:dc:39:6f:6a:69:80 Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option preference, len 1 Jul/31/2020 22:37:41: preference: 0 Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option DNS, len 16 Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option opt_86, len 16 Jul/31/2020 22:37:41: dhcp6_get_options: unknown or unexpected DHCP6 option opt_86, len 16 Jul/31/2020 22:37:41: dhcp6_get_options: get DHCP option IA_PD, len 41 Jul/31/2020 22:37:41: IA_PD: ID=1, T1=1800, T2=2880 Jul/31/2020 22:37:41: copyin_option: get DHCP option IA_PD prefix, len 25 Jul/31/2020 22:37:41: copyin_option: IA_PD prefix: 2001:16b8:45ca:b6c0::/60 pltime=3600 vltime=7200 Jul/31/2020 22:37:41: info_printf: nameserver[0] fd00::de39:6fff:fe6a:6980 Jul/31/2020 22:37:41: get_ia: make an IA: PD-1 Jul/31/2020 22:37:41: update_prefix: create a prefix 2001:16b8:45ca:b6c0::/60 pltime=3600, vltime=7200 Jul/31/2020 22:37:41: ifaddrconf: add an address 2001:16b8:45ca:b6c2::1/64 on enx0e504ad43426 Jul/31/2020 22:37:41: dhcp6_remove_event: removing an event on eno1, state=REQUEST Jul/31/2020 22:37:41: dhcp6_remove_event: removing server (ID: 00:03:00:01:dc:39:6f:6a:69:80) Jul/31/2020 22:37:41: client6_recvreply: got an expected reply, sleeping. ```

So we got the prefix 2001:16b8:45ca:b6c0::/60. We should also start radvd on the interface. We can use the ::/64 prefix to automatically select a prefix from the /60 we got:

sudo dist/tools/radvd/radvd.sh -c enx0e504ad43426 ::/64

ifconfig afterwards

``` eno1: flags=4163 mtu 1500 inet 192.168.42.69 netmask 255.255.255.0 broadcast 192.168.42.255 inet6 fe80::13ae:a2d4:3402:38fe prefixlen 64 scopeid 0x20 inet6 2001:16b8:45ca:b600:c732:1a76:eedb:8035 prefixlen 64 scopeid 0x0 ether 14:da:e9:4c:6b:2e txqueuelen 1000 (Ethernet) RX packets 1365523 bytes 1512790982 (1.5 GB) RX errors 0 dropped 62554 overruns 0 frame 0 TX packets 556281 bytes 73155386 (73.1 MB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 20 memory 0xf7f00000-f7f20000 enx0e504ad43426: flags=4163 mtu 1500 inet 169.254.1.2 netmask 255.255.0.0 broadcast 169.254.255.255 inet6 fe80::8520:5adc:6f1c:36f4 prefixlen 64 scopeid 0x20 inet6 2001:16b8:45ca:b6c2::1 prefixlen 64 scopeid 0x0 inet6 2001:16b8:45ca:b6c2:e8c6:3805:9c87:6c26 prefixlen 64 scopeid 0x0 ether 0e:50:4a:d4:34:26 txqueuelen 1000 (Ethernet) RX packets 9 bytes 816 (816.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 117 bytes 21513 (21.5 KB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions ```

Our border router should now have configured an address on it's upstream interface via router advertisements.

ifconfig RIOT

``` 2020-07-31 22:41:38,679 # Iface 8 HWaddr: 0E:50:4A:D4:34:25 2020-07-31 22:41:38,684 # L2-PDU:1500 MTU:1500 HL:64 RTR 2020-07-31 22:41:38,687 # Source address length: 6 2020-07-31 22:41:38,690 # Link type: wired 2020-07-31 22:41:38,695 # inet6 addr: fe80::c50:4aff:fed4:3425 scope: link VAL 2020-07-31 22:41:38,703 # inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 scope: global VAL 2020-07-31 22:41:38,705 # inet6 group: ff02::2 2020-07-31 22:41:38,708 # inet6 group: ff02::1 2020-07-31 22:41:38,712 # inet6 group: ff02::1:ffd4:3425 2020-07-31 22:41:38,713 # 2020-07-31 22:41:38,720 # Iface 7 HWaddr: 71:A7 Channel: 26 Page: 0 NID: 0x23 PHY: O-QPSK 2020-07-31 22:41:38,721 # 2020-07-31 22:41:38,725 # Long HWaddr: 5A:06:59:C9:2C:36:7D:7A 2020-07-31 22:41:38,732 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4 2020-07-31 22:41:38,739 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR 2020-07-31 22:41:38,742 # RTR_ADV 6LO IPHC 2020-07-31 22:41:38,745 # Source address length: 8 2020-07-31 22:41:38,748 # Link type: wireless 2020-07-31 22:41:38,754 # inet6 addr: fe80::5806:59c9:2c36:7d7a scope: link VAL 2020-07-31 22:41:38,757 # inet6 group: ff02::2 2020-07-31 22:41:38,759 # inet6 group: ff02::1 2020-07-31 22:41:38,763 # inet6 group: ff02::1:ff36:7d7a ```

To get a prefix on the downstream interface, we have to enable the DHCPv6 server

make BOARD=samr21-xpro UPLINK=cdc-ecm USE_DHCPV6=1 IPV6_PREFIX=2001:16b8:45ca:b6c0::/61 PORT= term

new ifconfig

``` 2020-07-31 22:46:17,746 # Iface 8 HWaddr: 0E:50:4A:D4:34:25 2020-07-31 22:46:17,751 # L2-PDU:1500 MTU:1500 HL:64 RTR 2020-07-31 22:46:17,754 # Source address length: 6 2020-07-31 22:46:17,757 # Link type: wired 2020-07-31 22:46:17,765 # inet6 addr: fe80::c50:4aff:fed4:3425 scope: link VAL 2020-07-31 22:46:17,770 # inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 scope: global VAL 2020-07-31 22:46:17,773 # inet6 group: ff02::2 2020-07-31 22:46:17,775 # inet6 group: ff02::1 2020-07-31 22:46:17,779 # inet6 group: ff02::1:ffd4:3425 2020-07-31 22:46:17,780 # 2020-07-31 22:46:17,787 # Iface 7 HWaddr: 71:A7 Channel: 26 Page: 0 NID: 0x23 PHY: O-QPSK 2020-07-31 22:46:17,788 # 2020-07-31 22:46:17,792 # Long HWaddr: 5A:06:59:C9:2C:36:7D:7A 2020-07-31 22:46:17,799 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4 2020-07-31 22:46:17,806 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR 2020-07-31 22:46:17,809 # RTR_ADV 6LO IPHC 2020-07-31 22:46:17,812 # Source address length: 8 2020-07-31 22:46:17,815 # Link type: wireless 2020-07-31 22:46:17,821 # inet6 addr: fe80::5806:59c9:2c36:7d7a scope: link VAL 2020-07-31 22:46:17,828 # inet6 addr: 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a scope: global VAL 2020-07-31 22:46:17,831 # inet6 group: ff02::2 2020-07-31 22:46:17,834 # inet6 group: ff02::1 2020-07-31 22:46:17,840 # inet6 group: ff02::1:ff36:7d7a 2020-07-31 22:46:17,841 # > ping6 2600:: 2020-07-31 22:46:26,583 # ping6 2600:: 2020-07-31 22:46:26,724 # 12 bytes from 2600::: icmp_seq=0 ttl=49 time=135.199 ms 2020-07-31 22:46:27,724 # 12 bytes from 2600::: icmp_seq=1 ttl=49 time=133.682 ms 2020-07-31 22:46:28,726 # 12 bytes from 2600::: icmp_seq=2 ttl=49 time=134.162 ms 2020-07-31 22:46:28,726 # 2020-07-31 22:46:28,729 # --- 2600:: PING statistics --- 2020-07-31 22:46:28,734 # 3 packets transmitted, 3 packets received, 0% packet loss 2020-07-31 22:46:28,739 # round-trip min/avg/max = 133.682/134.347/135.199 ms 2020-07-31 22:51:22,226 # nib neigh 2020-07-31 22:51:22,233 # fe80::8520:5adc:6f1c:36f4 dev #8 lladdr 0E:50:4A:D4:34:26 router REACHABLE GC 2020-07-31 22:51:22,241 # 2001:16b8:45ca:b6c2:fec2:3d00:0:bb1 dev #7 lladdr FC:C2:3D:00:00:00:0B:B1 STALE REGISTERED ```

Expected results

6LoWPAN clients got a global address and can reach other global addresses.

Actual results

Clients get a global address, but packets are not routed

2020-07-31 22:48:16,109 # Iface  7  HWaddr: 0B:B1  Channel: 26  Page: 0  NID: 0x23  PHY: O-QPSK 
2020-07-31 22:48:16,110 #           
2020-07-31 22:48:16,115 #           Long HWaddr: FC:C2:3D:00:00:00:0B:B1 
2020-07-31 22:48:16,121 #            TX-Power: 0dBm  State: IDLE  max. Retrans.: 3  CSMA Retries: 4 
2020-07-31 22:48:16,128 #           AUTOACK  ACK_REQ  CSMA  L2-PDU:102  MTU:1280  HL:64  RTR  
2020-07-31 22:48:16,131 #           RTR_ADV  6LO  IPHC  
2020-07-31 22:48:16,134 #           Source address length: 8
2020-07-31 22:48:16,137 #           Link type: wireless
2020-07-31 22:48:16,143 #           inet6 addr: fe80::fec2:3d00:0:bb1  scope: link  VAL
2020-07-31 22:48:16,151 #           inet6 addr: 2001:16b8:45ca:b6c2:fec2:3d00:0:bb1  scope: global  VAL
2020-07-31 22:48:16,154 #           inet6 group: ff02::2
2020-07-31 22:48:16,157 #           inet6 group: ff02::1
2020-07-31 22:48:16,160 #           inet6 group: ff02::1:ff00:bb1
2020-07-31 22:48:16,160 #           
2020-07-31 22:48:16,163 #           Statistics for Layer 2
2020-07-31 22:48:16,166 #             RX packets 4  bytes 286
2020-07-31 22:48:16,172 #             TX packets 54 (Multicast: 52)  bytes 2364
2020-07-31 22:48:16,175 #             TX succeeded 54 errors 0
2020-07-31 22:48:16,178 #           Statistics for IPv6
2020-07-31 22:48:16,181 #             RX packets 3  bytes 304
2020-07-31 22:48:16,186 #             TX packets 54 (Multicast: 52)  bytes 3488
2020-07-31 22:48:16,190 #             TX succeeded 54 errors 0
2020-07-31 22:48:16,191 # 
> ping6 2600::
2020-07-31 22:48:20,111 #  ping6 2600::
2020-07-31 22:48:23,113 # 
2020-07-31 22:48:23,114 # --- 2600:: PING statistics ---
2020-07-31 22:48:23,121 # 3 packets transmitted, 0 packets received, 100% packet loss
> ping6 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a
2020-07-31 22:49:39,113 #  ping6 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a
2020-07-31 22:49:39,134 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=0 ttl=64 rssi=-18 dBm time=10.944 ms
2020-07-31 22:49:40,135 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=1 ttl=64 rssi=-18 dBm time=12.000 ms
2020-07-31 22:49:41,134 # 12 bytes from 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a: icmp_seq=2 ttl=64 rssi=-18 dBm time=10.720 ms
2020-07-31 22:49:41,135 # 
2020-07-31 22:49:41,140 # --- 2001:16b8:45ca:b6c2:5806:59c9:2c36:7d7a PING statistics ---
2020-07-31 22:49:41,144 # 3 packets transmitted, 3 packets received, 0% packet loss
2020-07-31 22:49:41,150 # round-trip min/avg/max = 10.720/11.221/12.000 ms
> ping6 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425
2020-07-31 22:49:45,321 #  ping6 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425
2020-07-31 22:49:48,323 # 
2020-07-31 22:49:48,326 # --- 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425 PING statistics ---
2020-07-31 22:49:48,333 # 3 packets transmitted, 0 packets received, 100% packet loss
> nib neigh
2020-07-31 22:50:09,942 #  nib neigh
2020-07-31 22:50:09,949 # fe80::5806:59c9:2c36:7d7a dev #7 lladdr 5A:06:59:C9:2C:36:7D:7A router STALE GC

Instead a neighbor solicitation for the global address is performed.

Packet Capture

Versions

miri64 commented 4 years ago

Can you reproduce this with ethos + uhcp? Because that worked two weeks ago.

miri64 commented 4 years ago

what does nib route and nib neigh say for the respective nodes.

benpicco commented 4 years ago

Can you reproduce this with ethos + uhcp? Because that worked two weeks ago.

Will try.

what does nib route and nib neigh say for the respective nodes.

border router

2020-07-31 23:57:54,746 #  nib route
2020-07-31 23:57:54,749 # 2001:16b8:45ca:b6c2::/64 dev #7
2020-07-31 23:57:54,752 # 2001:16b8:45ca:b6c2::/64 dev #8
2020-07-31 23:57:54,755 # default* via fe80::1 dev #8
2020-07-31 23:58:07,369 #  nib neigh
2020-07-31 23:58:07,378 # 2001:16b8:45ca:b6c2:fec2:3d00:0:bb1 dev #7 lladdr FC:C2:3D:00:00:00:0B:B1  STALE REGISTERED
2020-07-31 23:58:07,383 # fe80::1 dev #8 lladdr 0E:50:4A:D4:34:26 router STALE GC
2020-07-31 23:58:07,390 # fe80::8520:5adc:6f1c:36f4 dev #8 lladdr 0E:50:4A:D4:34:26  STALE GC

node

2020-07-31 23:58:18,028 #  nib route
2020-07-31 23:58:18,031 # 2001:16b8:45ca:b6c2::/64 dev #7
2020-07-31 23:58:18,035 # default* via fe80::5806:59c9:2c36:7d7a dev #7
2020-07-31 23:58:20,972 #  nib neigh
2020-07-31 23:58:20,980 # fe80::5806:59c9:2c36:7d7a dev #7 lladdr 5A:06:59:C9:2C:36:7D:7A router STALE GC

miri64 commented 4 years ago

2020-07-31 23:57:54,749 # 2001:16b8:45ca:b6c2::/64 dev #7
2020-07-31 23:57:54,752 # 2001:16b8:45ca:b6c2::/64 dev #8

That's odd... why is the same prefix routed to two interfaces?

benpicco commented 4 years ago

That's odd... why is the same prefix routed to two interfaces?

hmm might have set it up wrong?

29: enx0e504ad43426: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 0e:50:4a:d4:34:26 brd ff:ff:ff:ff:ff:ff
    inet 169.254.1.2/16 brd 169.254.255.255 scope link noprefixroute enx0e504ad43426
       valid_lft forever preferred_lft forever
    inet6 2001:16b8:45ca:b6c2::1/64 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 2001:16b8:45ca:b6c2:e8c6:3805:9c87:6c26/64 scope global dynamic noprefixroute 
       valid_lft 86097sec preferred_lft 14097sec
    inet6 fe80::1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::8520:5adc:6f1c:36f4/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

I suppose I need to carve out a different prefix from the /60 for enx0e504ad43426 than what I delegate to DHCPv6

miri64 commented 4 years ago

Yeah, I was just about to ask for the routes set on the ethernet interface.

ip -6 route show dev enx0e504ad43426

There seems to be a 64 subnet configured for that, while the border router routes a 61 bit subnet.

benpicco commented 4 years ago

Hm so when I use IPV6_PREFIX=2001:16b8:45ca:b6c7::/62 for the downstream DHCPv6 / Kea, would I expect that to work?

2020-08-01 00:20:03,891 # Iface  8  HWaddr: 0E:50:4A:D4:34:25 
2020-08-01 00:20:03,896 #           L2-PDU:1500  MTU:1500  HL:64  RTR  
2020-08-01 00:20:03,900 #           Source address length: 6
2020-08-01 00:20:03,902 #           Link type: wired
2020-08-01 00:20:03,908 #           inet6 addr: fe80::c50:4aff:fed4:3425  scope: link  VAL
2020-08-01 00:20:03,915 #           inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425  scope: global  VAL
2020-08-01 00:20:03,918 #           inet6 group: ff02::2
2020-08-01 00:20:03,921 #           inet6 group: ff02::1
2020-08-01 00:20:03,925 #           inet6 group: ff02::1:ffd4:3425
2020-08-01 00:20:03,925 #           
2020-08-01 00:20:03,932 # Iface  7  HWaddr: 71:A7  Channel: 26  Page: 0  NID: 0x23  PHY: O-QPSK 
2020-08-01 00:20:03,933 #           
2020-08-01 00:20:03,937 #           Long HWaddr: 5A:06:59:C9:2C:36:7D:7A 
2020-08-01 00:20:03,944 #            TX-Power: 0dBm  State: IDLE  max. Retrans.: 3  CSMA Retries: 4 
2020-08-01 00:20:03,951 #           AUTOACK  ACK_REQ  CSMA  L2-PDU:102  MTU:1280  HL:64  RTR  
2020-08-01 00:20:03,954 #           RTR_ADV  6LO  IPHC  
2020-08-01 00:20:03,958 #           Source address length: 8
2020-08-01 00:20:03,960 #           Link type: wireless
2020-08-01 00:20:03,966 #           inet6 addr: fe80::5806:59c9:2c36:7d7a  scope: link  VAL
2020-08-01 00:20:03,974 #           inet6 addr: 2001:16b8:45ca:b6c7:5806:59c9:2c36:7d7a  scope: global  VAL
2020-08-01 00:20:03,976 #           inet6 group: ff02::2
2020-08-01 00:20:03,980 #           inet6 group: ff02::1
2020-08-01 00:20:03,983 #           inet6 group: ff02::1:ff36:7d7a
2020-08-01 00:20:03,984 #           
nib routeg
2020-08-01 00:20:06,880 #  nib route
2020-08-01 00:20:06,883 # 2001:16b8:45ca:b6c2::/64 dev #8
2020-08-01 00:20:06,886 # 2001:16b8:45ca:b6c7::/64 dev #7
2020-08-01 00:20:06,890 # default* via fe80::8520:5adc:6f1c:36f4 dev #8

ip -6 route show dev enx0e504ad43426

2001:16b8:45ca:b6c2::/64 proto ra metric 104 pref medium
2001:16b8:45ca:b6c2::/64 proto kernel metric 256 pref medium
2001:16b8:45ca:b6c4::/62 via fe80::2 metric 1024 pref medium
fe80::/64 proto kernel metric 104 pref medium
fe80::/64 proto kernel metric 256 pref medium

miri64 commented 4 years ago

Hm so when I use IPV6_PREFIX=2001:16b8:45ca:b6c7::/62 for the downstream DHCPv6 / Kea, would I expect that to work?

No it should be a smaller subnet, so either 63 or 64.

miri64 commented 4 years ago

We should also start radvd on the interface. We can use the ::/64 prefix to automatically select a prefix from the /60 we got:
sudo dist/tools/radvd/radvd.sh -c enx0e504ad43426 ::/64

Shouldn't the radvd also then advertise the 62-bit prefix to the border router?

benpicco commented 4 years ago

With IPV6_PREFIX=2001:16b8:45ca:b6c7::/63 I get

2020-08-01 00:23:18,089 # Iface  8  HWaddr: 0E:50:4A:D4:34:25 
2020-08-01 00:23:18,093 #           L2-PDU:1500  MTU:1500  HL:64  RTR  
2020-08-01 00:23:18,097 #           Source address length: 6
2020-08-01 00:23:18,099 #           Link type: wired
2020-08-01 00:23:18,105 #           inet6 addr: fe80::c50:4aff:fed4:3425  scope: link  VAL
2020-08-01 00:23:18,112 #           inet6 addr: 2001:16b8:45ca:b6c2:c50:4aff:fed4:3425  scope: global  VAL
2020-08-01 00:23:18,115 #           inet6 group: ff02::2
2020-08-01 00:23:18,118 #           inet6 group: ff02::1
2020-08-01 00:23:18,122 #           inet6 group: ff02::1:ffd4:3425
2020-08-01 00:23:18,123 #           
2020-08-01 00:23:18,129 # Iface  7  HWaddr: 71:A7  Channel: 26  Page: 0  NID: 0x23  PHY: O-QPSK 
2020-08-01 00:23:18,130 #           
2020-08-01 00:23:18,135 #           Long HWaddr: 5A:06:59:C9:2C:36:7D:7A 
2020-08-01 00:23:18,142 #            TX-Power: 0dBm  State: IDLE  max. Retrans.: 3  CSMA Retries: 4 
2020-08-01 00:23:18,150 #           AUTOACK  ACK_REQ  CSMA  L2-PDU:102  MTU:1280  HL:64  RTR  
2020-08-01 00:23:18,152 #           RTR_ADV  6LO  IPHC  
2020-08-01 00:23:18,155 #           Source address length: 8
2020-08-01 00:23:18,157 #           Link type: wireless
2020-08-01 00:23:18,164 #           inet6 addr: fe80::5806:59c9:2c36:7d7a  scope: link  VAL
2020-08-01 00:23:18,171 #           inet6 addr: 2001:16b8:45ca:b6c7:5806:59c9:2c36:7d7a  scope: global  VAL
2020-08-01 00:23:18,174 #           inet6 group: ff02::2
2020-08-01 00:23:18,176 #           inet6 group: ff02::1
2020-08-01 00:23:18,182 #           inet6 group: ff02::1:ff36:7d7a
2020-08-01 00:23:18,183 #           
nib route
2020-08-01 00:23:20,669 #  nib route
2020-08-01 00:23:20,672 # 2001:16b8:45ca:b6c7::/64 dev #7
2020-08-01 00:23:20,675 # 2001:16b8:45ca:b6c2::/64 dev #8
2020-08-01 00:23:20,678 # default* via fe80::1 dev #8

2001:16b8:45ca:b6c2::/64 proto ra metric 104 pref medium
2001:16b8:45ca:b6c2::/64 proto kernel metric 256 pref medium
2001:16b8:45ca:b6c6::/63 via fe80::2 metric 1024 pref medium
fe80::/64 proto kernel metric 104 pref medium
fe80::/64 proto kernel metric 256 pref medium

and still no routing from clients

miri64 commented 4 years ago

Let's get back a few steps and understand the problem. You have basically 3 sub nets here here

LAN
--------------------------------------
Host machine - 6LBR connection (H6LBR)
--------------------------------------
WPAN

and they have the following hierarchy LAN > H6LBR > WPAN. I.e. the prefix of the outer sub-net needs to be contained in every inner sub-net.

The LAN provides you with a 60-bit prefix A and you want at most a 64-bit prefix for the WPAN, so the H6LBR needs to be configured as such, that it has a prefix length in-between.

This means for your desired setup to work your host

has to have a global address with a prefix B of length 61 to 63 configured on the Ethernet interface to the border router,
has to advertise that same prefix via router advertisements to the border router, and
has to configure a DHCPv6 server or agent with prefix delegation for a prefix C of length 62 to 64 (so whatever length you used for the Ethernet interface + at least 1).

This should result in

prefix A being configured on the same interface as your host's default route
the host having a route to the prefix B via the link-local address of the Ethernet interface of the border router
prefix B being configured on the Ethernet interface of the border router (check with nib route or nib prefix) and having the default route set via the link-local address of the host's Ethernet interface
prefix C being configured on the WPAN interface, nib route goes to no link-local address.

If you apply this configuration do you get that result and is it then still not working?

benpicco commented 4 years ago

Thank you for the heads-up! I've changed sla-len to 1 in dhcp6c.conf so it will only shift the 'incoming' prefix by 1 to configure the USB-Ethernet device

Aug/02/2020 17:54:22: update_prefix: create a prefix 2001:16b8:453f:1fc0::/60 pltime=3600, vltime=7200 Aug/02/2020 17:54:22: ifaddrconf: add an address 2001:16b8:453f:1fc2::1/61 on enx0e504ad43426

Together with dist/tools/radvd/radvd.sh -c enx0e504ad43426 ::/61 this gives me

5: enx0e504ad43426: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
    link/ether 0e:50:4a:d4:34:26 brd ff:ff:ff:ff:ff:ff
    inet6 2001:16b8:453f:1fc2::1/61 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::1/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::5852:2e16:20ac:182f/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

and

2001:16b8:453f:1fc0::/62 via fe80::2 metric 1024 pref medium
2001:16b8:453f:1fc0::/61 proto ra metric 101 pref medium
2001:16b8:453f:1fc0::/61 proto kernel metric 256 pref medium
fe80::/64 proto kernel metric 101 pref medium
fe80::/64 proto kernel metric 256 pref medium

So I'm using IPV6_PREFIX=2001:16b8:453f:1fc2::/62 for gnrc_border_router which results in

2020-08-02 18:38:07,324 # Iface  8  HWaddr: 0E:50:4A:D4:34:25 
2020-08-02 18:38:07,329 #           L2-PDU:1500  MTU:1500  HL:64  RTR  
2020-08-02 18:38:07,333 #           Source address length: 6
2020-08-02 18:38:07,335 #           Link type: wired
2020-08-02 18:38:07,341 #           inet6 addr: fe80::c50:4aff:fed4:3425  scope: link  VAL
2020-08-02 18:38:07,348 #           inet6 addr: 2001:16b8:453f:1fc0:c50:4aff:fed4:3425  scope: global  VAL
2020-08-02 18:38:07,351 #           inet6 group: ff02::2
2020-08-02 18:38:07,354 #           inet6 group: ff02::1
2020-08-02 18:38:07,357 #           inet6 group: ff02::1:ffd4:3425
2020-08-02 18:38:07,358 #           
2020-08-02 18:38:07,365 # Iface  7  HWaddr: 71:A7  Channel: 26  Page: 0  NID: 0x23  PHY: O-QPSK 
2020-08-02 18:38:07,366 #           
2020-08-02 18:38:07,370 #           Long HWaddr: 5A:06:59:C9:2C:36:7D:7A 
2020-08-02 18:38:07,377 #            TX-Power: 0dBm  State: IDLE  max. Retrans.: 3  CSMA Retries: 4 
2020-08-02 18:38:07,389 #           AUTOACK  ACK_REQ  CSMA  L2-PDU:102  MTU:1280  HL:64  RTR  
2020-08-02 18:38:07,389 #           RTR_ADV  6LO  IPHC  
2020-08-02 18:38:07,391 #           Source address length: 8
2020-08-02 18:38:07,393 #           Link type: wireless
2020-08-02 18:38:07,399 #           inet6 addr: fe80::5806:59c9:2c36:7d7a  scope: link  VAL
2020-08-02 18:38:07,406 #           inet6 addr: 2001:16b8:453f:1fc2:5806:59c9:2c36:7d7a  scope: global  VAL
2020-08-02 18:38:07,409 #           inet6 group: ff02::2
2020-08-02 18:38:07,412 #           inet6 group: ff02::1
2020-08-02 18:38:07,416 #           inet6 group: ff02::1:ff36:7d7a
2020-08-02 18:38:07,417 #           
2020-08-02 18:38:09,337 #  nib route
2020-08-02 18:38:09,340 # 2001:16b8:453f:1fc0::/61 dev #8
2020-08-02 18:38:09,343 # 2001:16b8:453f:1fc2::/64 dev #7
2020-08-02 18:38:09,347 # default* via fe80::5852:2e16:20ac:182f dev #8
2020-08-02 18:39:50,264 #  nib prefix
2020-08-02 18:39:50,270 # 2001:16b8:453f:1fc0::/61 dev #8  expires 86224 sec deprecates 14224 sec
2020-08-02 18:39:50,277 # 2001:16b8:453f:1fc2::/64 dev #7  expires 39858 sec deprecates 29858 sec

in RIOT. Unfortunately this does not result in the border router being able to reach global addresses.

(if you know a better way to set this up let me know. I initially thought dhcp6relay would do what I wanted, but that sends DHCP RELAY messages that encapsulate the messages from RIOT's DHCP client and my Fritz!Box doesn't understand that)

miri64 commented 4 years ago

Unfortunately this does not result in the border router being able to reach global addresses.

But that is a different issue, isn't it? The original text was about forwarding the messages of host nodes (not clients, we are not on the application layer) within the WPAN, if I read it correctly, right? Not about the border router being unable to reach global addresses.

miri64 commented 4 years ago

I will try this myself btw if I find the time :-)

benpicco commented 4 years ago

But that is a different issue, isn't it?

I'm afraid it might as well be an issue with my setup.

miri64 commented 4 years ago

Still worth checking out IMHO. Since you are unsure yourself, however, I "degrade" the issue from a bug report to a question :-).

edgaralb commented 4 years ago

Hello

There is a guide to configure the border-router for openmote-b, subGHz.

best regards

benpicco commented 4 years ago

@edgaralb If you use UHCP (which is the default) only one interface will be provided with a prefix, so if you only want to use Sub-GHz you can add

DISABLE_MODULE += at86rf215_24ghz

to the Makefile.

If you want to use both interfaces, you'll have to switch to DHCPv6 instead. For this, simpl add USE_DHCPV6=1 to your command line or the Makefile. The script will automatically try to install a DHCPv6 server.

edgaralb commented 4 years ago

An additional question: In the latest RIOT release, there are only two modulation MR-O-QPSK and MR-OFDM, to use MR-FSK which library or when it will be added to the release

benpicco commented 4 years ago

I have it in a branch, I should just PR it already :flushed: I still have to verify that CCA and ACK timings are now correct.

Mind you that there is still no notion of 'FSK modes', you can just set any combination of options that the chip supports. The same is true for MR-O-QPSK and MR-OFDM, although there the option space is much smaller.

Another layer that verifies the configuration is valid according to the 802.15.4 spec / complies with local regulations is still needed. The upcoming submac seems to be a good fit.

edgaralb commented 4 years ago

Thank benpicco, I will review your commens.

benpicco commented 4 years ago

@edgaralb there is now a PR for MR-FSK: #14959

edgaralb commented 4 years ago

Thank you @benpicco . Your previous comments on the at86rf215 driver work great in openmote-b. I'm going to test the MR-FSK integration.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

RIOT-OS / RIOT