Closed mightyguava closed 11 months ago
More complete logs from journalctl -x
below:
This part specifically
Sep 19 02:30:08 mirror kernel: [WLAN_RFKILL]: rockchip_wifi_power: 0
Sep 19 02:30:08 mirror kernel: [WLAN_RFKILL]: rockchip_wifi_power: toggle = false
Sep 19 02:30:08 mirror kernel: wifi power off
Sep 19 02:30:08 mirror systemd[1]: Found device /sys/subsystem/net/devices/wlan0.
░░ Subject: A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully.
░░
░░ The job identifier is 78.
Sep 19 02:30:08 mirror systemd[1]: Started WPA supplicant for netplan wlan0.
░░ Subject: A start job for unit netplan-wpa-wlan0.service has finished successfully
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░
░░ A start job for unit netplan-wpa-wlan0.service has finished successfully.
░░
░░ The job identifier is 77.
Sep 19 02:30:08 mirror wpa_supplicant[603]: Successfully initialized wpa_supplicant
Sep 19 02:30:08 mirror wpa_supplicant[603]: Could not set interface wlan0 flags (UP): Device or resource busy
Sep 19 02:30:08 mirror wpa_supplicant[603]: nl80211: Could not set interface 'wlan0' UP
Sep 19 02:30:08 mirror wpa_supplicant[603]: nl80211: deinit ifname=wlan0 disabled_11b_rates=0
Sep 19 02:30:08 mirror wpa_supplicant[603]: Could not set interface wlan0 flags (UP): Device or resource busy
Sep 19 02:30:08 mirror wpa_supplicant[603]: WEXT: Could not set interface 'wlan0' UP
Sep 19 02:30:08 mirror wpa_supplicant[603]: wlan0: Failed to initialize driver interface
Looks like something is powering off wifi right before wpa_supplicant tries to run?
I wifi should be configured with the network config file in the boot partition before the first boot.
Also, if you install the desktop version, do you have a wifi issue?
As in I should put my netplan YAML changes in /boot/firmware/network-config
instead? I didn’t see anything in the wiki so I just found some instructions on Google. The use of cloud-init confused me too. What is that being used for?
Will try desktop tomorrow but I’d really like to get server working as I want to give the display device directly to an application instead of the desktop.
I wifi should be configured with the network config file in the boot partition before the first boot.
Ah I understand now, you are explaining what that header is about. My config is correct looking at https://github.com/Joshua-Riek/ubuntu-rockchip/issues/283.
So after a couple restarts I actually did manage to get wifi working at boot, same netplan
config file. For conciseness, here's journalctl -xe | grep wlan0
for the successful run
Sep 19 09:07:29 mirror kernel: [dhd] Register interface [wlan0] MAC: b8:2d:28:42:2a:f4
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] wl_android_wifi_off : in g_wifi_on=1, on_failure=1
Sep 19 09:07:29 mirror systemd[1]: Found device /sys/subsystem/net/devices/wlan0.
░░ Subject: A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully
░░ A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully.
Sep 19 09:07:29 mirror systemd[1]: Started WPA supplicant for netplan wlan0.
░░ Subject: A start job for unit netplan-wpa-wlan0.service has finished successfully
░░ A start job for unit netplan-wpa-wlan0.service has finished successfully.
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] wl_android_wifi_off : out
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] dhd_open : Enter
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : in g_wifi_on=0
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : Success
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] dhd_open : Exit ret=0
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] dhd_pri_open : tx queue started
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] custom_xps_map_set : Done. mapping cpu
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Link UP
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Link DOWN
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Re-configuring with /run/systemd/network/10-netplan-wlan0.network
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Link UP
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Gained carrier
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: Lost carrier
Sep 19 09:07:30 mirror systemd-networkd[620]: wlan0: DHCPv6 lease lost
Sep 19 09:07:32 mirror systemd-networkd[620]: wlan0: Gained IPv6LL
Sep 19 09:07:33 mirror wpa_supplicant[606]: wlan0: Trying to associate with SSID 'my-ssid'
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_ext_set_chanspec : channel 5g-161(0xe39b)
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_conn_debug_info : Connecting with a0:36:bc:38:86:dc ssid "my-ssid", len (4), channel=5g-161(chan_cnt=1), sec=wpa2/psk/mfpn/aes, rssi=-31
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_iw_event : Link UP with a0:36:bc:38:86:dc
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_ext_iapsta_event : [S] Link UP with a0:36:bc:38:86:dc
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_bss_connect_done : Report connect result - connection succeeded
Sep 19 09:07:33 mirror wpa_supplicant[606]: wlan0: Associated with a0:36:bc:38:86:dc
Sep 19 09:07:33 mirror wpa_supplicant[606]: wlan0: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
Sep 19 09:07:33 mirror kernel: [dhd] [wlan0] wl_add_keyext : key index (0)
Sep 19 09:07:33 mirror wpa_supplicant[606]: wlan0: WPA: Key negotiation completed with a0:36:bc:38:86:dc [PTK=CCMP GTK=CCMP]
Sep 19 09:07:33 mirror wpa_supplicant[606]: wlan0: CTRL-EVENT-CONNECTED - Connection to a0:36:bc:38:86:dc completed [id=0 id_str=]
Sep 19 09:07:33 mirror systemd-networkd[620]: wlan0: Gained carrier
Sep 19 09:07:34 mirror systemd-networkd[620]: wlan0: DHCPv4 address 192.168.1.132/24 via 192.168.1.1
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | wlan0 | True | 192.168.1.132 | 255.255.255.0 | global | b8:2d:28:42:2a:f4 |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | wlan0 | True | fe80::ba2d:28ff:fe42:2af4/64 | . | link | b8:2d:28:42:2a:f4 |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 0 | 0.0.0.0 | 192.168.1.1 | 0.0.0.0 | wlan0 | UG |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 1 | 192.168.1.0 | 0.0.0.0 | 255.255.255.0 | wlan0 | U |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 2 | 192.168.1.1 | 0.0.0.0 | 255.255.255.255 | wlan0 | UH |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 3 | 192.168.1.5 | 0.0.0.0 | 255.255.255.255 | wlan0 | UH |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 4 | 192.168.1.10 | 0.0.0.0 | 255.255.255.255 | wlan0 | UH |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 1 | fe80::/64 | :: | wlan0 | U |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 3 | local | :: | wlan0 | U |
Sep 19 09:09:30 mirror cloud-init[675]: ci-info: | 4 | multicast | :: | wlan0 | U |
And a subsequent restart that failed to connect to wifi, with no config changes.
Sep 19 09:11:21 mirror kernel: [dhd] Register interface [wlan0] MAC: b8:2d:28:42:2a:f4
Sep 19 09:11:21 mirror kernel: [dhd] [wlan0] wl_android_wifi_off : in g_wifi_on=1, on_failure=1
Sep 19 09:11:21 mirror systemd[1]: Found device /sys/subsystem/net/devices/wlan0.
░░ Subject: A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully
░░ A start job for unit sys-subsystem-net-devices-wlan0.device has finished successfully.
Sep 19 09:11:21 mirror systemd[1]: Started WPA supplicant for netplan wlan0.
░░ Subject: A start job for unit netplan-wpa-wlan0.service has finished successfully
░░ A start job for unit netplan-wpa-wlan0.service has finished successfully.
Sep 19 09:11:21 mirror wpa_supplicant[591]: Could not set interface wlan0 flags (UP): Device or resource busy
Sep 19 09:11:21 mirror wpa_supplicant[591]: nl80211: Could not set interface 'wlan0' UP
Sep 19 09:11:21 mirror wpa_supplicant[591]: nl80211: deinit ifname=wlan0 disabled_11b_rates=0
Sep 19 09:11:21 mirror wpa_supplicant[591]: Could not set interface wlan0 flags (UP): Device or resource busy
Sep 19 09:11:21 mirror wpa_supplicant[591]: WEXT: Could not set interface 'wlan0' UP
Sep 19 09:11:21 mirror wpa_supplicant[591]: wlan0: Failed to initialize driver interface
Sep 19 09:11:21 mirror wpa_supplicant[591]: wlan0: CTRL-EVENT-DSCP-POLICY clear_all
Sep 19 09:11:21 mirror systemd[1]: netplan-wpa-wlan0.service: Main process exited, code=exited, status=255/EXCEPTION
░░ An ExecStart= process belonging to unit netplan-wpa-wlan0.service has exited.
Sep 19 09:11:21 mirror systemd[1]: netplan-wpa-wlan0.service: Failed with result 'exit-code'.
░░ The unit netplan-wpa-wlan0.service has entered the 'failed' state with result 'exit-code'.
Sep 19 09:11:21 mirror kernel: [dhd] [wlan0] wl_android_wifi_off : out
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] dhd_open : Enter
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : in g_wifi_on=0
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : Success
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] dhd_open : Exit ret=0
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] dhd_pri_open : tx queue started
Sep 19 09:11:22 mirror kernel: [dhd] [wlan0] custom_xps_map_set : Done. mapping cpu
Sep 19 09:11:22 mirror systemd-networkd[606]: wlan0: Link UP
Sep 19 09:11:23 mirror systemd-networkd[606]: wlan0: Gained carrier
Sep 19 09:11:24 mirror systemd-networkd[606]: wlan0: Gained IPv6LL
Sep 19 09:13:22 mirror cloud-init[658]: ci-info: | wlan0 | True | fe80::ba2d:28ff:fe42:2af4/64 | . | link | b8:2d:28:42:2a:f4 |
Sep 19 09:13:22 mirror cloud-init[658]: ci-info: | 1 | fe80::/64 | :: | wlan0 | U |
Sep 19 09:13:22 mirror cloud-init[658]: ci-info: | 3 | local | :: | wlan0 | U |
Sep 19 09:13:22 mirror cloud-init[658]: ci-info: | 4 | multicast | :: | wlan0 | U |
It sure seems like there's a race with something occupying wlan0
the same time that netplan-wpa-wlan0.service
tries to initialize it.
The suspect being this block
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] wl_android_wifi_off : out
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] dhd_open : Enter
Sep 19 09:07:29 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : in g_wifi_on=0
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] wl_android_wifi_on : Success
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] dhd_open : Exit ret=0
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] dhd_pri_open : tx queue started
Sep 19 09:07:30 mirror kernel: [dhd] [wlan0] custom_xps_map_set : Done. mapping cpu
If it appears before wpa_supplicant
, wifi works on boot. If it appears after, wifi doesn't work on boot as wpa_supplicant
can't access wlan0
.
Do you know what this is and why the ubuntu image is toggling "android_wifi"?
Following instructions from ChatGPT 🤖 , I ran
systemctl edit netplan-wpa-wlan0.service
and added
[Service]
ExecStartPre=/bin/sleep 1
to add a 1 second delay before wpa_supplicant
. Now my wifi starts consistently across reboots.
The remaining issue is that systemd-networkd-wait-online
still thinks network isn't available and waits the full 2 minutes, even though you can clearly see that wlan0 is getting a DHCP address, and NTP is working.
Logs below
I ended up changing the timeout for systemd-networkd-wait-online
to 10 seconds instead of the default 2 minutes. There's nothing in the logs as to why it's failing. Any idea where to file a bug against?
Neither the 1 second delay in netplan-wpa-wlan0.service
nor lowering the timeout for the network wait feels right nor satisfying...
Thanks for putting some time into this, I don't have a lot of time right now, I'd like to make sure wifi is working normally.
Now I do know that the 2 minute wait is a normal Ubuntu 22.04 thing (however, i find it very annoying, if its safe to set it to 10 seconds i will probably set that as the default configuration). It used to be 5 minutes in Ubuntu 20.04
I did try to edit the delay to 10 seconds over the weekend but did not spend much time on it. Did you just set a timeout in the systemd service?
Yeah I did
sudo systemctl edit --full systemd-networkd-wait-online.service
And edited the command to
ExecStart=/usr/lib/systemd/systemd-networkd-wait-online --timeout=10
been stable across a few restarts.
Theoretically this service is supposed to exit successfully as soon as one of the interfaces connects to internet. Does it always hang and fail after 2 minutes for you as well?
I did some short testing, and the 2-minute time-out does not happen on my Orange Pi 5B. However, I encountered this on a different device over the weekend, so I know this exists. I don't have time to dig deeper right now, so I can't provide much information.
I made a change to overwrite the default link-local address https://github.com/Joshua-Riek/ubuntu-rockchip/commit/6eda35739053c6ac02bdf39e754e5fbc7a0c3a91. Maybe this will help.
build: https://github.com/Joshua-Riek/ubuntu-rockchip/actions/runs/6254729259
Oohhhh that makes sense. Great catch. I do not have ipv6.
I switched over to the Ubuntu desktop version and WiFi has been working fine. Getting some more sd cards in a couple days. I’ll flash and test your fix then.
I flashed your new image and double checked that link local setting. Still getting a 2 minute wait on start.
Whats the output of the command sudo networkctl
?
Also, have you connected to ethernet?
ubuntu@ubuntu:~$ sudo networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 eth0 ether routable configured
I was on wifi only earlier. Now I'm on ethernet. Still see the 2min timeout. Weirdly when I connect to ethernet, the wifi device fails to initialize.
Okay, now my wlan0 device is refusing to get created at all even without internet. Haven't made any config changes since the last update. It works just fine with the desktop image still so don't think it's a hardware issue.
This is reaaally weird, I don't see dhd
in my logs anywhere. dmesg | grep dhd
is empty. Not in journalctl
either. Is that like my install got corrupted?
Okay, now my wlan0 device is refusing to get created at all even without internet. Haven't made any config changes since the last update. It works just fine with the desktop image still so don't think it's a hardware issue.
I think you downloaded the Orange Pi 5 image and not the 5B image.
I have been trying to reproduce this on my side with no luck yet.
I actually just went and re-flashed the SD card half an hour ago and still have not been getting it to work. Ran this
sudo dd bs=4194304 if=ubuntu-22.04.3-preinstalled-server-arm64-orangepi-5b.img of=/dev/disk4
Put the sd card in. No wlan0
showing up in /sys/class/net
.
Gotta go grab dinner. That's probably all the time I have for the day, will look more tomorrow.
The weirdest part is that wifi worked on the first boot with your new image. But it hasn’t worked afterwards even after re-imaging
Managed to get my wlan0
interface to show up again: the dhd
stuff and wlan0
don't show up if I have my Anker USB hub (which has a keyboard & mouse attached) into the lower (closer to the board) of 2 USB-3 sockets. It's fine connected to the upper one, and connecting the keyboard by itself is fine in either socket. I suppose it's probably some obscure driver bug I'm hitting.
Diffing the dmesg between the good & bad runs, I see these in the bad run
+rk-pcie fe190000.pcie: PCIe Link Fail
+rk-pcie fe190000.pcie: failed to initialize host
which seems to lead to the rest of the failures.
I thought about this maybe being a power draw issue, but the hub is only powering a keyboard & mouse... and I don't have any of these problems when booting into the desktop image... I assume the destkop image has the same drivers and kernel as the server image...?
I would take all and any kernel messages with a grain of salt. The kernel is an absolute hackjob of a mess, but since I can not reproduce this, I would say your power supply is bad. Or your router has some weird internet configuration.
The power supply being suspect makes sense, but I'm having the same issue with both a brand new OrangePi 5V 4A charger as well as an Anker 65W PD charger. Router issues shouldn't cause the wlan0
interface to not initialize.
My router is OPNSense running a standard configuration, no VLANs or other fancy stuff, just DHCPv4.
Anyways, here's the networkctl
output you asked for running your new image.
ubuntu@ubuntu:~$ sudo networkctl
IDX LINK TYPE OPERATIONAL SETUP
1 lo loopback carrier unmanaged
2 eth0 ether routable configured
3 wlan0 wlan off unmanaged
Still had to wait for that 2 minutes...
I'm not sure what I can do since I can not reproduce this issue on my end.
Now, I could add the below as default, but I fear this could cause issues with other network configurations.
ExecStart=/usr/lib/systemd/systemd-networkd-wait-online --timeout=10
Outside of the pcie weirdness, I think there were 2 concrete things:
Add an override for netplan-wpa-wlan0.service
[Service]
ExecStartPre=/bin/sleep 1
And the timeout=10 you had above yeah. Thanks for helping me debug these issues!
Of course, but to confirm, is wifi working for you without any modification?
No.
WiFi does not work on boot or restarts. I have to manually run systemctl restart systemd-networkd
or netplan apply
after the system boots.
/bin/sleep 1
override to netplan-wpa-wlan0.service
WiFi is working after boot, but blocks 2 minutes on systemd-networkd-wait-online
--timeout=10
to systemd-networkd-wait-online
WiFi works after boot, but blocks 10 seconds on systemd-networkd-wait-online
.
Have not encountered any issues
I did some testing on my Orange Pi 5B and could not reproduce this issue. WiFi works as expected on the server image. My guess is your network-config is not setup correctly.
My network-config was posted at the start of this issue and I have not changed it. Since it’s a timing issue I wonder if my install being on emmc might contribute to it. I’m happy to move on though as there are workarounds and things are working now. Thanks for your help!
Hey, I'm sorry you're still having an issue with this, but I can not reproduce it as it works fine on my end. So I will be closing this.
I have the same issue on my Orange Pi 5B. Running netplan apply
after boot connects just fine to my wifi, but it does not connect at boot.
edit: I "solved" this issue by adding @reboot /usr/sbin/netplan apply
to my crontab. Really hope to find out what causes this issue though.
Brand new Orange Pi 5B. I burned the ubuntu-22.04.3-preinstalled-server-arm64-orangepi-5b.img.xz to eMMC.
The only changes I've made so far are to configure netplan and run an
apt dist-upgrade
.At startup, I get a message that says it's waiting for network ready and waits for about 2 minutes.
After that boot completes, and I get a link-local IPv6 address but no IPv4 address.
Running
will get wifi to connect almost immediately.
/etc/netplan/50-cloud-init.yaml
is configured withNot sure why this header is there but cloudinit has not been overwriting it across reboots.
Found these error logs but not sure what to do about it. Timestamp 02:30 is boot time, and then timestamp 02:34:43 is when I manually ran
systemctl restart systemd-networkd
It seems something is competing with wpa_supplicant for wlan0 on boot?