search

espressif / esp-lwip

Fork of lwIP (https://savannah.nongnu.org/projects/lwip/) with ESP-IDF specific patches
Other
79 stars 126 forks source link

NAPT routing not working between Wired ethernet and Wifi Station (IDFGH-3894) #20

Closed pmarches closed 3 years ago

pmarches commented 3 years ago

I am building a Wifi Extender with LWIP/NAPT. I am using a Olimex PoE board with ESP32. I can get the board to use both the Wired Ethernet (Static IP) and Wifi (Station mode). The idea is to build a PoE Wifi booster that will connect to a Wifi network, and use NAPT to forward traffic with the Wired interface. If I setup my Wifi adapter in both Station and AP mode, LWIP/NAPT forwards traffic correctly. But I am experiencing problems when trying to forward traffic between the Wifi adapter and the Wired Ethernet adapter.

Here is my network setup: Hostname IP Ethernet Gateway
ClientWired 192.168.11.1 4c:11:ae:d7:05:f3 gateway 192.168.11.2
ESP32Wired 192.168.11.2 80:fa:5b:5c:fe:76 gateway 192.168.11.1
ESP32Wifi 192.168.0.113 4c:11:ae:d7:05:f0 gateway 192.168.0.1
ServerWifi 192.168.0.100 38:f9:d3:49:f6:70 gateway 192.168.0.1
AccessPointWifi 192.168.0.1

The test command I use

ServerWifi> nc -lp 9876
ClientWired> nc 192.168.0.100 9876

Using Wireshark, I see the SYN packet come in on ServerWifi, then the ACK packet goes from ServerWifi to ESP32Wifi. But then ESP32Wifi returns a RST packet (Connection Reset) instead of forwarding the packet to ESP32Wired.

I turned on traces in LWIP and I see the PCB to be empty. I am not very familiar with LWIP, but I suspect the active PCB should contain something.

Active PCB states:
Listen PCB states:
TIME-WAIT PCB states:

Here is a sample of the relevant traces;

tcpip_thread: PACKET 0x3ffc311c
pbuf_remove_header: old 0x3ffca1d0 new 0x3ffca1de (14)
ip4_input_accept: #2 iphdr->dest 0x7100a8c0 netif->ip_addr 0x7100a8c0 (0xa8c0, 0xa8c0, 0x71000000)
(netif_is_up(netif)=1
ip4_addr_isany_val(*netif_ip4_addr(netif))=0
ip4_input: packet accepted on interface st #2
ip4_input: 
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        64     | (v, hl, tos, len)
+-------------------------------+
|        0      |010|       0   | (id, flags, offset)
+-------------------------------+
|   64  |    6  |    0xb892     | (ttl, proto, chksum)
+-------------------------------+
|  192  |  168  |    0  |  100  | (src)
+-------------------------------+
|  192  |  168  |    0  |  113  | (dest)
+-------------------------------+
ip4_input: p->len 64 p->tot_len 64
pbuf_remove_header: old 0x3ffca1de new 0x3ffca1f2 (20)
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:134
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:142
TCP header:
+-------------------------------+
|     9876      |    61265      | (src port, dest port)
+-------------------------------+
|           -1203661195          | (seq no)
+-------------------------------+
|           1662591131          | (ack no)
+-------------------------------+
| 11 |   |010010|     65535     | (hdrlen, flags (SYN ACK 
), win)
+-------------------------------+
|    0x44e1     |         0     | (chksum, urgp)
+-------------------------------+
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:144
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:164
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:187
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:193
pbuf_remove_header: old 0x3ffca1f2 new 0x3ffca21e (44)
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:246
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:248
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:261
Active PCB states:
Listen PCB states:
TIME-WAIT PCB states:
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:306
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:410
+-+-+-+-+-+-+-+-+-+-+-+-+-+- tcp_input: flags SYN ACK 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:426
TRACING tcp_input /home/pmarches/projects/marine_is/esp32/esp-idf/components/lwip/lwip/src/core/tcp_in.c:613
tcp_input: no PCB match found, resetting.
pbuf_alloc(length=20)
pbuf_alloc(length=20) == 0x3ffca224
pbuf_add_header: old 0x3ffca274 new 0x3ffca260 (20)
ip4_output_if: st2
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        40     | (v, hl, tos, len)
+-------------------------------+
|        5      |000|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x39a5     | (ttl, proto, chksum)
+-------------------------------+
|  192  |  168  |    0  |  113  | (src)
+-------------------------------+
|  192  |  168  |    0  |  100  | (dest)
+-------------------------------+
ip4_output_if: call netif->output()
pbuf_add_header: old 0x3ffca260 new 0x3ffca252 (14)
pbuf_free(0x3ffca224)
pbuf_free: deallocating 0x3ffca224
tcp_rst: seqno 1662591131 ackno -1203661194.
pbuf_free(0x3ffca1b8)
pbuf_free: deallocating 0x3ffca1b8
D (23322) emac_esp32: receive len= 74
pbuf_alloc(length=74)
pbuf_alloc(length=74) == 0x3ffc9d68
david-cermak commented 3 years ago

Hi @pmarches

Thanks for the report! My guess is that the esp-idf ethernet port layer inputs packets to lwip by reference only and doesn't support the ESP_L2_TO_L3_COPY (option to plain memcpy input data from the driver layer to the stack) the same way as WiFi interface.

Could you please test your setup with this patch applied to the esp-idf? 0001-lwip-Fix-ethernet-port-to-use-ESP_L2_TO_L3_COPY.patch.txt

pmarches commented 3 years ago

That totally worked! Amazing how I have spent 3 days adding traces and understanding how LWIP works, and then @david-cermak comes along and shoots from the hip a working patch.

Thanks!