search

IntergatedCircuits / IPoverUSB

STM32 lwIP networking via USB
Apache License 2.0
69 stars 16 forks source link

IPoverUSB doesn't work with Linux or Mac #1

Closed zenczykowski closed 3 years ago

zenczykowski commented 3 years ago

On Linux ethernet usb0 device shows up, but DHCP fails to get an answer. Similarly on Mac - device shows up, but DHCP fails to get an IP address.

Below are included relevant logs from a Fedora 32 Linux box (the pcap has .log extension since github doesn't want to allow .pcap).

This is a FreeRTOS build of IPoverUSB.

diff --git a/Makefile b/Makefile index 4a27072..aecde89 100644 --- a/Makefile +++ b/Makefile @@ -16,7 +16,7 @@ XPD_DIR = STM32_XPD USBD_DIR = USBDevice LWIPDIR = lwIP/src CONTRIBDIR = lwip-contrib -OS_DIR = +OS_DIR = FreeRTOS/FreeRTOS/Source

++---- Included files ----++

include $(LWIPDIR)/Filelists.mk

This is using a: https://www.amazon.com/gp/product/B00CW9AKDY/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1 [STM32F4DISCOVERY] ST STM32 STM32F4 STM32F407 MCU Discovery Evaluation Development Board kit embedded ST-LINK/V2 debugger @XYG

Best guess is Linux/Mac and the IPoverUSB device are not agreeing on USB URB (possibly MBIM) framing and thus frames don't flow successfully in either direction.

[ 0.000000] Linux version 5.8.9-200.fc32.x86_64 (mockbuild@bkernel01.iad2.fedoraproject.org) (gcc (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1), GNU ld version 2.34-4.fc32) #1 SMP Mon Sep 14 18:28:45 UTC 2020 ... [316478.147751] cdc_ncm 1-6:1.1 enp0s20f0u6i1: unregister 'cdc_ncm' usb-0000:00:14.0-6, CDC NCM [316484.539176] usb 1-6: new full-speed USB device number 7 using xhci_hcd [316484.666861] usb 1-6: New USB device found, idVendor=0483, idProduct=ffff, bcdDevice= 1.00 [316484.666873] usb 1-6: New USB device strings: Mfr=16, Product=32, SerialNumber=48 [316484.666881] usb 1-6: Product: IP over USB Demonstrator [316484.666886] usb 1-6: Manufacturer: STMicroelectronics [316484.666891] usb 1-6: SerialNumber: 19004100 [316484.690225] cdc_ncm 1-6:1.1: MAC-Address: 00:80:e1:00:00:00 [316484.690671] cdc_ncm 1-6:1.1 usb0: register 'cdc_ncm' at usb-0000:00:14.0-6, CDC NCM, 00:80:e1:00:00:00 [316484.708297] cdc_ncm 1-6:1.1 enp0s20f0u6i1: renamed from usb0 [316552.462093] device enp0s20f0u6i1 entered promiscuous mode [316552.463409] cdc_ncm 1-6:1.1 enp0s20f0u6i1: 10 mbit/s downlink 10 mbit/s uplink [316552.479401] cdc_ncm 1-6:1.1 enp0s20f0u6i1: network connection: connected [316552.479457] IPv6: ADDRCONF(NETDEV_CHANGE): enp0s20f0u6i1: link becomes ready [316619.602751] device enp0s20f0u6i1 left promiscuous mode

ip link set enp0s20f0u6i1 up; tcpdump -s 10000 -w dump.pcap -i enp0s20f0u6i1

tcpdump -r dump.pcap

reading from file dump.pcap, link-type EN10MB (Ethernet) dropped privs to tcpdump 17:07:47.493508 ARP, Request who-has 192.168.0.1 tell 192.168.0.1, length 28 17:07:47.495812 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 17:07:47.606866 IP6 :: > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 17:07:48.438819 IP6 :: > ff02::1:ff00:0: ICMP6, neighbor solicitation, who has zeus.lan, length 32 17:07:49.462910 IP6 zeus.lan > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 17:07:49.462926 IP6 zeus.lan > ff02::2: ICMP6, router solicitation, length 16 17:07:49.958865 IP6 zeus.lan > ff02::16: HBH ICMP6, multicast listener report v2, 1 group record(s), length 28 17:07:53.110865 IP6 zeus.lan > ff02::2: ICMP6, router solicitation, length 16 17:08:00.726869 IP6 zeus.lan > ff02::2: ICMP6, router solicitation, length 16 17:08:15.574873 IP6 zeus.lan > ff02::2: ICMP6, router solicitation, length 16 17:08:21.490129 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:80:e1:00:00:00 (oui Unknown), length 300 17:08:27.113025 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:80:e1:00:00:00 (oui Unknown), length 300 17:08:38.358633 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 00:80:e1:00:00:00 (oui Unknown), length 300 17:08:46.294864 IP6 zeus.lan > ff02::2: ICMP6, router solicitation, length 16

dhclient -4 -d -v enp0s20f0u6i1

Internet Systems Consortium DHCP Client 4.4.2b1 Copyright 2004-2019 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/enp0s20f0u6i1/00:80:e1:00:00:00 Sending on LPF/enp0s20f0u6i1/00:80:e1:00:00:00 Sending on Socket/fallback DHCPDISCOVER on enp0s20f0u6i1 to 255.255.255.255 port 67 interval 6 (xid=0xbb3dfe7c) DHCPDISCOVER on enp0s20f0u6i1 to 255.255.255.255 port 67 interval 11 (xid=0xbb3dfe7c) DHCPDISCOVER on enp0s20f0u6i1 to 255.255.255.255 port 67 interval 18 (xid=0xbb3dfe7c) ^C dmesg.txt dump.pcap.log

zenczykowski commented 3 years ago

I've noticed something possibly interesting...

04:11:58.254377 BULK COMPLETE from 1:15:1 0x0000: 4e43 4d48 0c00 0000 5400 3800 ffff ffff NCMH....T.8..... 0x0010: ffff 0080 e100 0001 0806 0001 0800 0604 ................ 0x0020: 0001 0080 e100 0001 c0a8 0001 0000 0000 ................ 0x0030: 0000 c0a8 0001 0000 4e43 4d30 1000 0000 ........NCM0.... 0x0040: 0c00 2a00 0000 0000 0000 0000 0000 0000 ..*............. 0x0050: 0000 0000 .... 04:11:58.254381 BULK SUBMIT to 1:15:1 04:11:58.259289 BULK SUBMIT to 1:15:1 0x0000: 4e43 4d48 0c00 0000 c400 0c00 4e43 4d30 NCMH........NCM0 0x0010: 1000 0000 6a00 5a00 0000 0000 0000 0000 ....j.Z......... 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0060: 0000 0000 0000 0000 0000 3333 0000 0016 ..........33.... 0x0070: 0080 e100 0000 86dd 6000 0000 0024 0001 ........`....$.. 0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0090: ff02 0000 0000 0000 0000 0000 0000 0016 ................ 0x00a0: 3a00 0502 0000 0100 8f00 6f8a 0000 0001 :.........o..... 0x00b0: 0400 0000 ff02 0000 0000 0000 0000 0001 ................ 0x00c0: ff00 0000 ....

So it looks like header order differs between those sent by IPoverUSB and those sent by Linux. IPoverUSB does NCMH, packet, NCM0 while Linux does NCMH, NCM0, packet

zenczykowski commented 3 years ago

There seems to be other possible mistakes, but this one:

[maze@zeus USBDevice]$ git show commit 0ecb1907bca410a1d8936d5c393dfaf3f6567157 (HEAD) Author: Maciej Żenczykowski zenczykowski@gmail.com Date: Sat Sep 26 04:57:07 2020 -0700

x

diff --git a/Class/CDC/usbd_ncm.c b/Class/CDC/usbd_ncm.c index 23e0565..15b369c 100644 --- a/Class/CDC/usbd_ncm.c +++ b/Class/CDC/usbd_ncm.c @@ -565,7 +565,7 @@ static void ncm_outData(USBD_NCM_IfHandleType itf, USBD_EpHandleType ep) USBD_NCM_DatagramPointerTableType pt = (void)nth + ndpIndex;

     /* Verify all tables in advance */

makes dhcp work for Linux and makes ping 192.168.0.1 and dns work.

ip link set enp0s20f0u5i1 up; dhclient -4 -d -v enp0s20f0u5i1

Internet Systems Consortium DHCP Client 4.4.2b1 Copyright 2004-2019 Internet Systems Consortium. All rights reserved. For info, please visit https://www.isc.org/software/dhcp/

Listening on LPF/enp0s20f0u5i1/00:80:e1:00:00:00 Sending on LPF/enp0s20f0u5i1/00:80:e1:00:00:00 Sending on Socket/fallback DHCPDISCOVER on enp0s20f0u5i1 to 255.255.255.255 port 67 interval 8 (xid=0x843a5b51) DHCPDISCOVER on enp0s20f0u5i1 to 255.255.255.255 port 67 interval 13 (xid=0x843a5b51) DHCPOFFER of 192.168.0.2 from 192.168.0.1 DHCPREQUEST for 192.168.0.2 on enp0s20f0u5i1 to 255.255.255.255 port 67 (xid=0x843a5b51) DHCPACK of 192.168.0.2 from 192.168.0.1 (xid=0x843a5b51) bound to 192.168.0.2 -- renewal in 1642 seconds.

$ ip addr show dev enp0s20f0u5i1 13: enp0s20f0u5i1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 00:80:e1:00:00:00 brd ff:ff:ff:ff:ff:ff inet 192.168.0.2/24 brd 192.168.0.255 scope global dynamic enp0s20f0u5i1 valid_lft 3513sec preferred_lft 3513sec inet6 fe80::280:e1ff:fe00:0/64 scope link valid_lft forever preferred_lft forever

$ arping 192.168.0.1 ARPING 192.168.0.1 from 192.168.0.2 enp0s20f0u5i1 Unicast reply from 192.168.0.1 [00:80:E1:00:00:01] 2.234ms ^CSent 1 probes (1 broadcast(s)) Received 1 response(s)

$ ping 192.168.0.1 PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data. 64 bytes from 192.168.0.1: icmp_seq=1 ttl=255 time=1.75 ms ^C --- 192.168.0.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.751/1.751/1.751/0.000 ms

$ host www.lwip.home 192.168.0.1 Using domain server: Name: 192.168.0.1 Address: 192.168.0.1#53 Aliases:

www.lwip.home has address 192.168.0.1 www.lwip.home has address 192.168.0.1 www.lwip.home has address 192.168.0.1

zenczykowski commented 3 years ago 
diff --git a/Class/CDC/usbd_ncm.c b/Class/CDC/usbd_ncm.c
index 23e0565..15b369c 100644
--- a/Class/CDC/usbd_ncm.c
+++ b/Class/CDC/usbd_ncm.c
@@ -565,7 +565,7 @@ static void ncm_outData(USBD_NCM_IfHandleType *itf, USBD_EpHandleType *ep)
         USBD_NCM_DatagramPointerTableType* pt = (void*)nth + ndpIndex;

         /* Verify all tables in advance */
-        while ((ndpIndex > sizeof(nth->V16)) &&
+        while ((ndpIndex >= sizeof(nth->V16)) &&
                (ndpIndex < nth->V16.BlockLength) &&
                (pt->V16.Signature == ndp16_sign.dw) &&
                (pt->V16.Length > sizeof(pt->V16)))
benedekkupper commented 3 years ago

Thanks for digging into this, I've taken over your fix. I take it that the issue is fixed with this change, please let me know otherwise, and reopen the ticket.

Just a short explanation on why I'm putting the NDP at the end: This allows the application to keep adding datagrams to an NTB while the other one is being transferred. If the NDP were put at the beginning, we'd limit the maximum amount of datagrams that can fit in there, since the NDP size increases with each added datagram. Of course it would be possible to add a new NDP for each datagram that is added, but that's wasting more precious memory.

zenczykowski commented 3 years ago

Thank you - that should indeed be enough.

I'm not certain (I've not tried figuring it out from the code) but I think Linux (on transmit) just uses a statically sized NCMH + NCM0 buffer with space for a certain maximum number of datagrams...

That's presumably why there's so many extra 0s (between the end of the NCM0 and the actual datagram) in the following:

0x0000: 4e43 4d48 0c00 0000 c400 0c00 4e43 4d30 NCMH........NCM0 0x0010: 1000 0000 6a00 5a00 0000 0000 0000 0000 ....j.Z......... 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0040: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0050: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0060: 0000 0000 0000 0000 0000 3333 0000 0016 ..........33.... 0x0070: 0080 e100 0000 86dd 6000 0000 0024 0001 ........`....$.. 0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 ................ 0x0090: ff02 0000 0000 0000 0000 0000 0000 0016 ................ 0x00a0: 3a00 0502 0000 0100 8f00 6f8a 0000 0001 :.........o..... 0x00b0: 0400 0000 ff02 0000 0000 0000 0000 0001 ................ 0x00c0: ff00 0000 ....

The above has room for 0x54 == 84 bytes, ie. 20 (+ 1 EOF) datagrams if the NCM0 header was extended to the max.

(Anyway, not certain of this at all, it doesn't really matter... there could also be some alignment reasons or something)