Closed cweagans closed 1 year ago
You can find the expected dmesg boot output of the Pi 3 B+ at https://gist.github.com/gokrazy-bot/5bab202a89e1623f192292b9080d18a9
Notably, the brcmfmac lines should read:
[Raspberry Pi 3 B+] [ 9.262669] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[Raspberry Pi 3 B+] [ 9.273447] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,3-model-b-plus.bin failed with error -2
[Raspberry Pi 3 B+] [ 9.495633] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Mar 2 2020 23:30:41 version 7.45.202 (r724630 CY) FWID 01-72f6ece2
[Raspberry Pi 3 B+] [ 13.209249] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
If you boot Raspbian from an SD card, can you check the dmesg log messages? Does it load the firmware correctly on your device, and if so, which version?
Maybe we just need to pull in new firmware files, but I’m not sure why it works on my Pi 3 B+ but not on yours… :-/
Hmm, https://github.com/gokrazy/firmware/pull/299 is failing with a similar error message to yours, but I thought that would be triggered by the new firmware version (which isn’t merged yet).
Perhaps you could try reverting to an older kernel and/or firmware version? The easiest way would probably be to check out both the kernel and firmware repository locally, use a replace directive, then use git reset --hard
to go back to an earlier revision. See also https://gokrazy.org/development/modules/
Given that also a new kernel version update is failing with a similar error, my guess would be that Linux 6.2 made loading firmware flaky (hence it passed CI once, but doesn’t pass any more). I’ll try to take a look some time next week.
I'll give those ideas a try, thanks!
I'm not sure how much activity there normally is on the related kernel modules, but it seems that there has been a fair amount of change in the last month: https://github.com/search?q=repo%3Atorvalds%2Flinux+brcmfmac&type=commits&s=author-date&o=desc
I can reproduce the issue.
With kernel 6.1.12:
[ 5.717560] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[ 5.726623] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,3-model-b-plus.bin failed with error -2
[ 5.891060] Bluetooth: Core ver 2.22
[ 5.894848] NET: Registered PF_BLUETOOTH protocol family
[ 5.901401] Bluetooth: HCI device and connection manager initialized
[ 5.908049] Bluetooth: HCI socket layer initialized
[ 5.914489] Bluetooth: L2CAP socket layer initialized
[ 5.919746] Bluetooth: SCO socket layer initialized
[ 6.010086] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Mar 2 2020 23:30:41 version 7.45.202 (r724630 CY) FWID 01-72f6ece2
[ 6.040081] Bluetooth: HCI UART driver ver 2.3
[ 6.046692] Bluetooth: HCI UART protocol H4 registered
[ 6.052683] Bluetooth: HCI UART protocol Broadcom registered
[ 6.058534] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 6.069264] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[ 6.183520] uart-pl011 3f201000.serial: no DMA platform data
[ 6.436680] Bluetooth: hci0: BCM: chip id 107
[ 6.441542] Bluetooth: hci0: BCM: features 0x2f
[ 6.447789] Bluetooth: hci0: BCM4345C0
[ 6.451676] Bluetooth: hci0: BCM4345C0 (003.001.025) build 0000
[ 6.457912] Bluetooth: hci0: BCM: firmware Patch file not found, tried:
[ 6.464728] Bluetooth: hci0: BCM: 'brcm/BCM4345C0.raspberrypi,3-model-b-plus.hcd'
[ 6.472394] Bluetooth: hci0: BCM: 'brcm/BCM4345C0.hcd'
[ 6.477702] Bluetooth: hci0: BCM: 'brcm/BCM.raspberrypi,3-model-b-plus.hcd'
[ 6.484892] Bluetooth: hci0: BCM: 'brcm/BCM.hcd'
[ 7.872845] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 9.692293] IPv6: ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
/tmp/breakglass501445588 # uname -a
Linux gokrazy 6.1.12 #1 SMP PREEMPT Wed Mar 1 20:57:29 UTC 2017 aarch64 GNU/Linux
But then, the most recent commit (kernel 6.2) broke it:
[ 5.656470] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[ 5.665572] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,3-model-b-plus.bin failed with error -2
[ 5.817815] Bluetooth: Core ver 2.22
[ 5.822534] NET: Registered PF_BLUETOOTH protocol family
[ 5.828118] Bluetooth: HCI device and connection manager initialized
[ 5.835832] Bluetooth: HCI socket layer initialized
[ 5.841091] Bluetooth: L2CAP socket layer initialized
[ 5.846532] Bluetooth: SCO socket layer initialized
[ 5.929929] Bluetooth: HCI UART driver ver 2.3
[ 5.934689] Bluetooth: HCI UART protocol H4 registered
[ 5.940345] Bluetooth: HCI UART protocol Broadcom registered
[ 5.941303] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 5.946438] brcmfmac: brcmf_fwvid_request_module: mod=wcc: failed -2
[ 5.953953] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator
[ 5.960253] ieee80211 phy0: brcmf_attach: brcmf_fwvid_attach failed
[ 6.007983] brcmfmac: brcmf_sdio_firmware_callback: brcmf_attach failed
[ 6.085724] uart-pl011 3f201000.serial: no DMA platform data
[ 6.328510] Bluetooth: hci0: BCM: chip id 107
[ 6.333404] Bluetooth: hci0: BCM: features 0x2f
[ 6.339711] Bluetooth: hci0: BCM4345C0
[ 6.343635] Bluetooth: hci0: BCM4345C0 (003.001.025) build 0000
[ 6.349891] Bluetooth: hci0: BCM: firmware Patch file not found, tried:
[ 6.356641] Bluetooth: hci0: BCM: 'brcm/BCM4345C0.raspberrypi,3-model-b-plus.hcd'
[ 6.364303] Bluetooth: hci0: BCM: 'brcm/BCM4345C0.hcd'
[ 6.369580] Bluetooth: hci0: BCM: 'brcm/BCM.raspberrypi,3-model-b-plus.hcd'
[ 6.376693] Bluetooth: hci0: BCM: 'brcm/BCM.hcd'
[ 7.862337] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
/tmp/breakglass3848229575 # uname -a
Linux gokrazy 6.2.0 #1 SMP PREEMPT Wed Mar 1 20:57:29 UTC 2017 aarch64 GNU/Linux
The kernel doesn’t pass CI, but there was a bug in the CI setup that incorrectly tested the live kernel version, not the version from the PR. This is now fixed with:
I haven’t found a report of the WiFi issue with Linux 6.2 yet.
Will try to bisect it to an individual 6.2 commit when I find some more time.
Now starting to bisect this issue. Let’s see if we can track it down.
Could be related to https://github.com/raspberrypi/linux/issues/5379
Could be related to raspberrypi/linux#5379
Thanks for the tip, but that issue is about kernel commit https://github.com/raspberrypi/linux/commit/22dfb21bf1cd876616d45cda1bc6daa89eec6747.
Our issue is caused by commit https://github.com/torvalds/linux/commit/d6a5c562214f26e442c8ec3ff1e28e16675d1bcf
I’ll send an email to the commit authors.
Informed the authors about the breakage in https://lore.kernel.org/linux-wireless/CANnVG6=a3etRagG+RaSEH-b4_nfzxpEKffQtuMWrttrbgjunZQ@mail.gmail.com/
Ah yeah 🤦 got confused sorry. I was debugging firmware issues with that one today, that's why I got mistaken :) Nevermind, glad you found the issue.
Turns out that commit splits up code into two kernel modules, and our wifi program didn’t load the new one!
With the fix https://github.com/gokrazy/wifi/commit/f611656338b68faa5f42bc2c8bbf1e0231a74a50, the new kernel version successfully passed CI: https://github.com/gokrazy/kernel/pull/500
Hooray! That's exciting -- definitely going to give this another go in the near future. I got tied up with my job hunt, so never circled back to this 😅
Awesome! Thanks for finding and fixing ✨
If I run
/user/wifi
, I get1969/12/31 17:20:50 no interfaces found
. I noticed that it was complaining about a missingbrcmfmac43455-sdio.raspberrypi,3-model-b-plus.bin
but https://github.com/raspberrypi/linux/issues/4248#issuecomment-1266751443 suggests that this warning is expected (for some reason)?Any ideas about where to start debugging?
config.json:
dmesg: