NTP from usb0 not working with H.1 firmware

tay-bird commented 1 year ago

Problem: I've followed the setup guides for Raspberry Pi hookup + software config, as well as the SBC guide for Network Time Protocol, but my chrony server doesn't report any clients and the Create3 log file doesn't show any attempts to get NTP data from it. The network appears to be set up correctly.

Components:

Create3 w/ H.1.0 firmware
Raspberry Pi 4 w/ Ubuntu Server 22.04

Setup process I followed:

Set connector switch to USB.
Connect rpi to create3 via usb-c cable.
SSH to raspberry pi.
Edit usercfg.txt and cmdline.txt as described.
Add usb0 in netplan with static address 192.168.186.3/24.
Install ros2 Humble.
Install chrony.

Add chrony config:

server 192.168.186.2 presend 0 minpoll 0 maxpoll 0 iburst  prefer trust
# Enable serving time to ntp clients on 192.168.186.0 subnet.
allow 192.168.186.0/24
local stratum 10

Reboot rpi and create3.
SSH to rpi.
Confirm create3 web server is reachable at 192.168.186.2:80

Output of chrony server:

$ sudo chronyc clients
Hostname                      NTP   Drop Int IntL Last     Cmd   Drop Int  Last
===============================================================================

create3 logs:

ar  7 00:17:01 socks user.notice network_manager:    PRODUCT         : create3
Mar  7 00:17:01 socks user.notice network_manager:    SERIAL          : e17628
Mar  7 00:17:01 socks user.notice network_manager:    CLEAN MODE      : normal
Mar  7 00:17:01 socks user.notice network_manager:    CONNS SVC MODE  : normal
Mar  7 00:17:01 socks user.notice network_manager:    AP RADIO        : wlan1/g/6/[SHORT-GI-20][GF]/Create-66FD/US
Mar  7 00:17:01 socks user.notice network_manager:    AP IP ADDR      : 192.168.10.1
Mar  7 00:17:01 socks user.notice network_manager:    CL IP           : dhcp/#/#/#
Mar  7 00:17:01 socks user.notice network_manager:    CL DNS          : #/#/#
Mar  7 00:17:01 socks user.notice network_manager:    CL DHCPCD T/O   : 3
Mar  7 00:17:01 socks user.notice network_manager:    CL REG DOMAIN   : WO
Mar  7 00:17:01 socks user.notice network_manager:    ETH0 IP         : static/192.168.186.2
Mar  7 00:17:01 socks user.notice network_manager:    USB0 IP         : 192.168.186.2/192.168.187.2/192.168.186.3
Mar  7 00:17:01 socks user.notice network_manager:    SYSTEM_ACCESS   : locked
Mar  7 00:17:01 socks user.notice network_manager:    DEVELOPMENT MODE: disabled
Mar  7 00:17:01 socks user.notice network_manager:    RADIO_LIST      : wlan0 wlan1
Mar  7 00:17:01 socks user.notice network_manager:    HOSTNAME        : socks
Mar  7 00:17:01 socks user.notice network_manager:    ROBOT ID        : 66FD42A02EE74A589F10583BF73AF4DE
Mar  7 00:17:01 socks user.notice network_manager:    BOOT COUNT      : 20
Mar  7 00:17:01 socks user.notice network_manager:    LANGUAGE PACK   : unknown
Mar  7 00:17:01 socks user.notice network_manager:    EXTRA LANGUAGE  : not_installed
Mar  7 00:17:02 socks user.notice network_manager: Provisioning of socks Complete

rpi networks:

ifconfig
lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 9832  bytes 28126074 (28.1 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 9832  bytes 28126074 (28.1 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.186.3  netmask 255.255.255.0  broadcast 192.168.186.255
        inet6 fe80::20fc:bdff:febe:1fb0  prefixlen 64  scopeid 0x20<link>
        ether 22:fc:bd:be:1f:b0  txqueuelen 1000  (Ethernet)
        RX packets 2021  bytes 210819 (210.8 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 3728  bytes 676015 (676.0 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.50.86  netmask 255.255.255.0  broadcast 192.168.50.255
        inet6 fe80::e65f:1ff:fec4:1149  prefixlen 64  scopeid 0x20<link>
        ether e4:5f:01:c4:11:49  txqueuelen 1000  (Ethernet)
        RX packets 29998  bytes 8913757 (8.9 MB)
        RX errors 0  dropped 1595  overruns 0  frame 0
        TX packets 35209  bytes 21680235 (21.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

shamlian commented 1 year ago

Could you, from your Raspberry Pi, curl -X POST http://192.168.186.2/api/restart-ntpd and post the last handful of lines of output of the robot's log after doing so? I confess I'm not sure what's wrong, but I'd like to try to help.

tay-bird commented 1 year ago

All I see in the logs is:

Mar  8 03:02:34 socks user.notice ntpd: Server 'ntpd' was stopped gracefully.

Here's a few other log lines that might be relevant:

Mar  8 02:59:40 socks user.notice ntpd: ntpd: bad address '0.irobot.pool.ntp.org'
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.062360 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.064565 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.074679 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.077917 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.083103 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 02:59:40 socks user.notice create-platform: 1678244380.093821 [69]        tev: ddsi_udp_conn_write to udp/192.168.50.86:35201 failed with retcode -1
Mar  8 03:02:23 socks daemon.err network[1468]: [ERROR] (src/wifi_control.c:207 errno:No child process) sudo network_device_interfaces_control.sh --interface=softap --action=del returned status 1

and:

Mar  8 03:13:52 socks user.notice create-platform: 1678245232.080862 [69]        tev: ddsi_udp_conn_write to udp/239.255.0.1:24650 failed with retcode -3
Mar  8 03:13:53 socks kern.err kernel: [  848.732397] usb 1-1: device descriptor read/64, error -71
Mar  8 03:13:53 socks kern.info kernel: [  849.054580] usb 1-1: New USB device found, idVendor=0525, idProduct=a4a2
Mar  8 03:13:53 socks kern.info kernel: [  849.054602] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Mar  8 03:13:53 socks kern.info kernel: [  849.054613] usb 1-1: Product: RNDIS/Ethernet Gadget
Mar  8 03:13:53 socks kern.info kernel: [  849.054623] usb 1-1: Manufacturer: Linux 5.15.0-1024-raspi with fe980000.usb

192.168.50.0/24 is the wifi router subnet.

tay-bird commented 1 year ago

Update: I've tried swapping to Galactic firmware and still seeing this - the rpi has a network connection to the create3 and has claimed 192.168.183.3, but the create3 never logs an NTP attempt back to 192.168.186.2.

shamlian commented 1 year ago

In case it is helpful: Clearpath has a new configuration file for the Create robot's NTP server that works with their use case; perhaps it can help to craft one that works with yours. https://github.com/turtlebot/turtlebot4/issues/216

shamlian commented 1 year ago

Checking in again -- users have had some success with this -- see #401 . Would be curious to know if this solves your problem as well.

iRobotEducation / create3_docs

NTP from usb0 not working with H.1 firmware #341