RIOT-OS / RIOT

RIOT - The friendly OS for IoT
https://riot-os.org
GNU Lesser General Public License v2.1
4.91k stars 1.98k forks source link

gnrc_border_router not routing between WiFi and 802.15.4 on esp32 #13005

Closed benpicco closed 4 years ago

benpicco commented 4 years ago

Description

I tried setting up a esp32 as a border router by hooking up an at86rf233 radio to it.

AFAIU the border router functionality only depends on the gnrc_sixlowpan_border_router_default module, so if I use that (and fib?) it should just work™

Maybe some additional steps are necessary, but I couldn't find them documented, so here is what I've tried:

Steps to reproduce the issue

Since the Makefile of examples/gnrc_border_router is a bit convoluted, I added the essential parts to examples/gnrc_networking

# this should include everything necessary 
USEMODULE += gnrc_sixlowpan_border_router_default

USEMODULE += at86rf233
USEMODULE += esp_wifi
GNRC_NETIF_NUMOF := 2

CFLAGS += -DESP_WIFI_SSID=\"36C3-insecure\"

# Specify the mandatory networking modules for 6LoWPAN border router
# Add forwarding table
USEMODULE += fib
at86rf233 config

```Makefile AT86RF2XX_PARAM_SPI = SPI_DEV\(0\) AT86RF2XX_PARAM_SPI_CLK = SPI_CLK_1MHZ AT86RF2XX_PARAM_CS ?= GPIO5 AT86RF2XX_PARAM_INT ?= GPIO4 AT86RF2XX_PARAM_RESET ?= GPIO21 AT86RF2XX_PARAM_SLEEP ?= GPIO2 CFLAGS += -DAT86RF2XX_PARAM_SPI=$(AT86RF2XX_PARAM_SPI) CFLAGS += -DAT86RF2XX_PARAM_SPI_CLK=$(AT86RF2XX_PARAM_SPI_CLK) CFLAGS += -DAT86RF2XX_PARAM_CS=$(AT86RF2XX_PARAM_CS) CFLAGS += -DAT86RF2XX_PARAM_INT=$(AT86RF2XX_PARAM_INT) CFLAGS += -DAT86RF2XX_PARAM_RESET=$(AT86RF2XX_PARAM_RESET) ```

small hack to get esp_wifi connect to unencrypted WiFi

```patch diff --git a/cpu/esp32/esp-wifi/esp_wifi_netdev.c b/cpu/esp32/esp-wifi/esp_wifi_netdev.c index 8429b01ef..e23f29867 100644 --- a/cpu/esp32/esp-wifi/esp_wifi_netdev.c +++ b/cpu/esp32/esp-wifi/esp_wifi_netdev.c @@ -251,7 +251,7 @@ static wifi_config_t wifi_config_sta = { .scan_method = WIFI_ALL_CHANNEL_SCAN, .sort_method = WIFI_CONNECT_AP_BY_SIGNAL, .threshold.rssi = -127, - .threshold.authmode = WIFI_AUTH_WPA_WPA2_PSK + .threshold.authmode = WIFI_AUTH_OPEN } }; ```

Expected results

The esp32 gets a global IP address on the WiFi interface and announces a route on the 802.15.4 interface.

Actual results

I get two interfaces as expected, but no global IP yet.

``` 2019-12-26 21:46:31,518 # Iface 10 HWaddr: 3C:71:BF:9E:13:FC Link: up 2019-12-26 21:46:31,521 # L2-PDU:1500 MTU:1500 HL:64 RTR 2019-12-26 21:46:31,525 # RTR_ADV Source address length: 6 2019-12-26 21:46:31,528 # Link type: wireless 2019-12-26 21:46:31,533 # inet6 addr: fe80::3e71:bfff:fe9e:13fc scope: link VAL 2019-12-26 21:46:31,536 # inet6 group: ff02::2 2019-12-26 21:46:31,539 # inet6 group: ff02::1 2019-12-26 21:46:31,542 # inet6 group: ff02::1:ff9e:13fc 2019-12-26 21:46:31,543 # 2019-12-26 21:46:31,546 # Statistics for Layer 2 2019-12-26 21:46:31,549 # RX packets 0 bytes 0 2019-12-26 21:46:31,554 # TX packets 1 (Multicast: 1) bytes 0 2019-12-26 21:46:31,556 # TX succeeded 0 errors 0 2019-12-26 21:46:31,559 # Statistics for IPv6 2019-12-26 21:46:31,562 # RX packets 0 bytes 0 2019-12-26 21:46:31,566 # TX packets 1 (Multicast: 1) bytes 64 2019-12-26 21:46:31,569 # TX succeeded 1 errors 0 2019-12-26 21:46:31,570 # 2019-12-26 21:46:31,574 # Iface 8 HWaddr: 5F:AA Channel: 26 Page: 0 NID: 0x23 2019-12-26 21:46:31,578 # Long HWaddr: AA:1F:52:9C:BD:30:DF:AA 2019-12-26 21:46:31,585 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4 2019-12-26 21:46:31,591 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR 2019-12-26 21:46:31,593 # RTR_ADV 6LO IPHC 2019-12-26 21:46:31,596 # Source address length: 8 2019-12-26 21:46:31,599 # Link type: wireless 2019-12-26 21:46:31,605 # inet6 addr: fe80::a81f:529c:bd30:dfaa scope: link VAL 2019-12-26 21:46:31,607 # inet6 group: ff02::2 2019-12-26 21:46:31,610 # inet6 group: ff02::1 2019-12-26 21:46:31,614 # inet6 group: ff02::1:ff30:dfaa 2019-12-26 21:46:31,615 # 2019-12-26 21:46:31,618 # Statistics for Layer 2 2019-12-26 21:46:31,621 # RX packets 7 bytes 301 2019-12-26 21:46:31,625 # TX packets 0 (Multicast: 0) bytes 0 2019-12-26 21:46:31,628 # TX succeeded 0 errors 0 2019-12-26 21:46:31,631 # Statistics for IPv6 2019-12-26 21:46:31,634 # RX packets 7 bytes 448 2019-12-26 21:46:31,638 # TX packets 0 (Multicast: 0) bytes 0 2019-12-26 21:46:31,641 # TX succeeded 0 errors 0 ```

I get a global IP promptly after pinging the broadcast address (that doesn't seem right)

``` 2019-12-26 21:49:29,046 # ping6 ff02::1 2019-12-26 21:49:29,066 # 12 bytes from fe80::5806:59c9:2c36:7d79: icmp_seq=0 ttl=64 rssi=-32 dBm time=11.471 ms 2019-12-26 21:49:29,084 # 12 bytes from fe80::3: icmp_seq=0 ttl=64 time=32.235 ms (DUP!) 2019-12-26 21:49:29,089 # 12 bytes from fe80::2: icmp_seq=0 ttl=64 time=32.876 ms (DUP!) 2019-12-26 21:49:30,063 # 12 bytes from fe80::5806:59c9:2c36:7d79: icmp_seq=1 ttl=64 rssi=-31 dBm time=8.489 ms 2019-12-26 21:49:30,070 # 12 bytes from fe80::2: icmp_seq=1 ttl=64 time=17.257 ms (DUP!) 2019-12-26 21:49:30,075 # 12 bytes from fe80::3: icmp_seq=1 ttl=64 time=17.675 ms (DUP!) 2019-12-26 21:49:31,065 # 12 bytes from fe80::5806:59c9:2c36:7d79: icmp_seq=2 ttl=64 rssi=-31 dBm time=10.655 ms 2019-12-26 21:49:31,065 # 2019-12-26 21:49:31,068 # --- ff02::1 PING statistics --- 2019-12-26 21:49:31,073 # 3 packets transmitted, 3 packets received, 4 duplicates, 0% packet loss 2019-12-26 21:49:31,078 # round-trip min/avg/max = 8.489/18.665/32.876 ms 2019-12-26 21:49:32,846 # ifconfig 2019-12-26 21:49:32,852 # Iface 10 HWaddr: 3C:71:BF:9E:13:FC Link: up 2019-12-26 21:49:32,856 # L2-PDU:1500 MTU:1500 HL:64 RTR 2019-12-26 21:49:32,860 # RTR_ADV Source address length: 6 2019-12-26 21:49:32,862 # Link type: wireless 2019-12-26 21:49:32,868 # inet6 addr: fe80::3e71:bfff:fe9e:13fc scope: link VAL 2019-12-26 21:49:32,875 # inet6 addr: 2001:67c:20a1:1192:3e71:bfff:fe9e:13fc scope: global VAL 2019-12-26 21:49:32,878 # inet6 group: ff02::2 2019-12-26 21:49:32,881 # inet6 group: ff02::1 2019-12-26 21:49:32,884 # inet6 group: ff02::1:ff9e:13fc 2019-12-26 21:49:32,885 # 2019-12-26 21:49:32,888 # Statistics for Layer 2 2019-12-26 21:49:32,891 # RX packets 7 bytes 546 2019-12-26 21:49:32,896 # TX packets 5 (Multicast: 5) bytes 276 2019-12-26 21:49:32,899 # TX succeeded 4 errors 0 2019-12-26 21:49:32,901 # Statistics for IPv6 2019-12-26 21:49:32,905 # RX packets 7 bytes 448 2019-12-26 21:49:32,909 # TX packets 5 (Multicast: 5) bytes 284 2019-12-26 21:49:32,912 # TX succeeded 5 errors 0 2019-12-26 21:49:32,913 # 2019-12-26 21:49:32,917 # Iface 8 HWaddr: 5F:AA Channel: 26 Page: 0 NID: 0x23 2019-12-26 21:49:32,922 # Long HWaddr: AA:1F:52:9C:BD:30:DF:AA 2019-12-26 21:49:32,928 # TX-Power: 0dBm State: IDLE max. Retrans.: 3 CSMA Retries: 4 2019-12-26 21:49:32,934 # AUTOACK ACK_REQ CSMA L2-PDU:102 MTU:1280 HL:64 RTR 2019-12-26 21:49:32,936 # RTR_ADV 6LO IPHC 2019-12-26 21:49:32,939 # Source address length: 8 2019-12-26 21:49:32,942 # Link type: wireless 2019-12-26 21:49:32,948 # inet6 addr: fe80::a81f:529c:bd30:dfaa scope: link VAL 2019-12-26 21:49:32,950 # inet6 group: ff02::2 2019-12-26 21:49:32,953 # inet6 group: ff02::1 2019-12-26 21:49:32,956 # inet6 group: ff02::1:ff30:dfaa 2019-12-26 21:49:32,957 # 2019-12-26 21:49:32,960 # Statistics for Layer 2 2019-12-26 21:49:32,964 # RX packets 13 bytes 538 2019-12-26 21:49:32,968 # TX packets 3 (Multicast: 3) bytes 93 2019-12-26 21:49:32,971 # TX succeeded 3 errors 0 2019-12-26 21:49:32,974 # Statistics for IPv6 2019-12-26 21:49:32,977 # RX packets 13 bytes 796 2019-12-26 21:49:32,982 # TX packets 3 (Multicast: 3) bytes 156 2019-12-26 21:49:32,984 # TX succeeded 3 errors 0 2019-12-26 22:01:29,559 # nib route 2019-12-26 22:01:29,564 # 2001:67c:20a1:1192::/64 dev #10 2019-12-26 22:01:29,566 # default* via fe80::1 dev #10 ```

But a nearby samr21-xpro that is just running plain gnrc_networking does not receive any routes.

Maybe #8796 is needed for automatic configuration? I tried continuing with manual configuration as described in the gnrc_border_router example, but documentation there is a bit sparse for anything other than TAP. (Will make a PR to expand it should I get it working)

Since I already have an address on the WiFi interface, I suppose I only need to add one for the 802.15.4 interface:

ifconfig 8 add 2001:67c:20a1:1192::3/64
fibroute add :: via fe80::3e71:bfff:fe9e:13fc dev 10

2019-12-26 22:03:30,439 #  fibroute
2019-12-26 22:03:30,452 # Destination                             Flags             Next Hop                                Flags      Expires          Interface
2019-12-26 22:03:30,463 # ::                                      0x00000000 H      fe80::3e71:bfff:fe9e:13fc               0x00000000 NEVER            10

The example says

This prefix will be automatically added by the motes in the radio range.

However, still no routes on the samr21-xpro and I can't ping the newly added global address on the 6lo interface. (I can ping the link-local address just fine)

Now I don't know if this is due to bugs in RIOT or a wrong configuration on my end, but I think a WiFi-802.15.4 border router would be a good application to have :)

Versions

RIOT master

miri64 commented 4 years ago

Some things unclear to me:

miri64 commented 4 years ago
* Why are you using `fib`, not the `ipv6_nib`?

Ohhh... the example still contains a USEMODULE += fib... Yeah that's not used anymore. Try using ip route [edit]nib route of course[/edit] instead.

miri64 commented 4 years ago

With #8796 merged I was able to coax the ESP32 to send DHCPv6 solicits with

this patch

diff --git a/examples/gnrc_border_router/Makefile b/examples/gnrc_border_router/Makefile
index 30fdec09e..d8b1ad0fa 100644
--- a/examples/gnrc_border_router/Makefile
+++ b/examples/gnrc_border_router/Makefile
@@ -10,15 +10,6 @@ RIOTBASE ?= $(CURDIR)/../..
 # The following boards do not have an available UART
 BOARD_BLACKLIST += pic32-wifire pic32-clicker ruuvitag thingy52

-# use ethos (ethernet over serial) for network communication and stdio over
-# UART, but not on native, as native has a tap interface towards the host.
-ifeq (,$(filter native,$(BOARD)))
-  # ethos baudrate can be configured from make command
-  ETHOS_BAUDRATE ?= 115200
-  CFLAGS += -DETHOS_BAUDRATE=$(ETHOS_BAUDRATE)
-else
-  TERMFLAGS += -z [::1]:17754
-endif
 GNRC_NETIF_NUMOF := 2

 # SLIP legacy compatibility
@@ -55,8 +46,8 @@ USEMODULE += shell
 USEMODULE += shell_commands
 USEMODULE += ps

-# include UHCP client
-USEMODULE += gnrc_uhcpc
+# include DHCPv6 client
+USEMODULE += gnrc_dhcpv6_client

 # Optionally include RPL as a routing protocol. When includede gnrc_uhcpc will
 # configure the node as a RPL DODAG root when receiving a prefix.
@@ -69,25 +60,6 @@ DEVELHELP ?= 1

 # Change this to 0 show compiler invocation lines by default:
 QUIET ?= 1
-TAP ?= tap0
-IPV6_PREFIX ?= 2001:db8::/64
-
-ifeq (native,$(BOARD))
-TERMDEPS += uhcpd-daemon
-
-.PHONY: uhcpd-daemon
-
-uhcpd-daemon: host-tools
-   $(RIOTTOOLS)/uhcpd/bin/uhcpd $(TAP) $(IPV6_PREFIX) &
-else
-# We override the `make term` command to use ethos
-TERMPROG ?= sudo sh $(RIOTTOOLS)/ethos/start_network.sh
-TERMFLAGS ?= $(PORT) $(TAP) $(IPV6_PREFIX)
-
-# We depend on the ethos host tools to run the border router, we build them
-# if necessary
-TERMDEPS += host-tools
-endif

 include $(RIOTBASE)/Makefile.include

diff --git a/examples/gnrc_border_router/Makefile.board.dep b/examples/gnrc_border_router/Makefile.board.dep
deleted file mode 100644
index 56cdaf5ed..000000000
--- a/examples/gnrc_border_router/Makefile.board.dep
+++ /dev/null
@@ -1,6 +0,0 @@
-# Put board specific dependencies here
-ifeq (,$(filter native,$(BOARD)))
-  USEMODULE += stdio_ethos
-else
-  USEMODULE += socket_zep
-endif
diff --git a/examples/gnrc_border_router/main.c b/examples/gnrc_border_router/main.c
index 654b411a5..f27460074 100644
--- a/examples/gnrc_border_router/main.c
+++ b/examples/gnrc_border_router/main.c
@@ -22,9 +22,36 @@

 #include "shell.h"
 #include "msg.h"
+#include "net/gnrc/netif.h"
+#include "net/dhcpv6/client.h"
+#include "net/sock.h"
+#include "thread.h"
+#include "xtimer.h"

 #define MAIN_QUEUE_SIZE     (8)
 static msg_t _main_msg_queue[MAIN_QUEUE_SIZE];
+static char _dhcpv6_client_stack[DHCPV6_CLIENT_STACK_SIZE];
+
+void *_dhcpv6_client_thread(void *args)
+{
+    event_queue_t event_queue;
+    gnrc_netif_t *netif = gnrc_netif_iter(NULL);
+
+    (void)args;
+    xtimer_sleep(5);
+    /* initialize client event queue */
+    event_queue_init(&event_queue);
+    /* initialize DHCPv6 client on any interface */
+    dhcpv6_client_init(&event_queue, SOCK_ADDR_ANY_NETIF);
+    /* configure client to request prefix delegation of /64 subnet 
+     * interface netif */
+    dhcpv6_client_req_ia_pd(netif->pid, 64U);
+    /* start DHCPv6 client */
+    dhcpv6_client_start();
+    /* start event loop of DHCPv6 client */
+    event_loop(&event_queue);   /* never returns */
+    return NULL;
+}

 int main(void)
 {
@@ -33,6 +60,10 @@ int main(void)
     msg_init_queue(_main_msg_queue, MAIN_QUEUE_SIZE);
     puts("RIOT border router example application");

+    /* start DHCPv6 client */
+    thread_create(_dhcpv6_client_stack, DHCPV6_CLIENT_STACK_SIZE,
+                  DHCPV6_CLIENT_PRIORITY, THREAD_CREATE_STACKTEST,
+                  _dhcpv6_client_thread, NULL, "dhcpv6-client");
     /* start shell */
     puts("All up, running the shell now");
     char line_buf[SHELL_DEFAULT_BUFSIZE];

I used esp_now and esp_wifi as the network devices, so esp_now is selected as the interface to search a prefix for. Instead of just using the first interface one could also search for the ESP-now/WPAN interface in _dhcpv6_client_thread().

miri64 commented 4 years ago

The DHCPv6 client on the node does not however accept the advertises from the server for some reason. See https://github.com/RIOT-OS/RIOT/pull/8796#issuecomment-570244161

miri64 commented 4 years ago

With the latest fix (which increases the receive buffer size for the DHCPv6 client) I was able to get a prefix and a route configured on my OpenWRT. The only thing left to do was to configure the default route:

nib route add 8 :: <WRT link-local address>
miri64 commented 4 years ago

Not sure, why the default route is not configured by the router advertisements of the node... maybe I my router just does not send them.

gschorcht commented 4 years ago

@benpicco Maybe you could try a slightly modified version of RIOT_ESP_NOW_WiFi_Border_Router? Changing theMakefile.include in this project to

 # platform specific modules and settings
 GNRC_NETIF_NUMOF := 2
 USEMODULE += esp_wifi
-USEMODULE += esp_now
+USEMODULE += mrf24j40

worked out of the box for me with 6LoWPAN. It uses 2001:db8::/64 as prefix for the 6LoWPAN while the ESP32 border router gets GUA with a prefix from the router, 2003:c1:e703:555::/64 in my example. Once I set the route to 2001:db8::/64 on my local machine with

sudo ip -6 route add 2001:db8::/64 via 2003:c1:e703:555:260a:c4ff:fe06:a164 dev eth0

I can ping the 6LoWPAN node.

If you want the 6LoWPAN network to be part of the global routing prefix you get from your ISP, you would have to assign a subnet of this prefix to your BR at the 6LoWPAN interface. You would also have to tell your router that this part of the network is reachable via the WiFi interface of the BR, which might be a bit tricky.

For example, my ISP provides a prefix of 56 bits. 2003:c1:e703:500::/56. The router uses 2003:c1:e703:555::/64 for the LAN. The BR also gets a GUA in this network, for example 2003:c1:e703:555:260a:c4ff:fe06:a164. I could use 2003:c1:e703:556::/64 for the 6LoWPAN network by defining IPV6_PREFIX='2003:c1:e703:556' during the compilation. However, I would also have to tell my router that the 2003:c1:e703:556::/64 subnet is reachable via the GUA of the WiFi interface of the border router. Because the prefix for the 6LoWPAN network is compiled in, it will not work for prefixes that are changed by the ISP.

benpicco commented 4 years ago

Sorry I should close the issue since this is resolved with DHCPv6 merged now. The border router application still needs an update for this (#8803 - but this won't work for WiFi <-> 802.15.4 since it assumes the uplink to be wired) but that's a small change.

I had it working already when I tested the DHCPv6 PR.