Bluetooth LE connection issue (coexistence problem?)

[IB1387]

Hello!

We are developing an industrial IoT device with bluetooth LE and wifi support using the Compute Module 4 with the built-in RF module. BLE is used to connect to multiple sub-devices via GATT. We upload payloads, telemetry information and do device management tasks over the internet using the built-in wifi or ethernet of the CM4.

We experience problems, where our device sometimes fails to initiate any BLE connection, when it's connected to a 2.4GHz wifi network. This issue might be related to !12, though we are not sure and we didn't want to pollute the discussion there. Turning off wifi or switching to a 5GHz network drastically reduces failed connections. The issue does not show up 100% of the time, but we experience quite often that 99% of BLE connection attempts fail for multiple days at a time. Scanning for devices, seems to work ok regardless if connected to a 2.4GHz wifi network or not.

We tried different versions of RaspberryPiOS and even Ubuntu. We also tried to update the kernel to no effect. Multiple versions of bluez showed the same issue.

The BLE connection is of upmost importance for our use case. We stumbled upon some (not publicly documented?) bluetooth/wifi coexistence parameters e.g. btc_mode online. Setting btc_mode=4 in brcmfmac config currently seems to alleviate the issue at least partially, though we're still early in the testing phase of this solution.

Is there a public documentation of wifi/bt driver options and their values, especially the ones that correspond to their coexistence? Can someone help us find the optimal combination of settings for our use case?

Thanks in advance!

---

Detailed information about our setup:

• Compute Module 4 with built-in flash and wifi/bt
• Custom carrier PCB based on the CM4 IO board design
• Stock RaspberryPiOS lite arm64 (2022-04-04)
• Kernel 5.15.32-v8+
• BlueZ 5.55
• wpa_supplicant v2.9

The issue was replicated with different versions of RaspberryPiOS (and even Ubuntu), different kernel versions, a original CM4 IO board instead of our custom board as well as different wpa_supplicant and BlueZ versions.

Because it was asked in the other thread:

user@raspberrypi:~ $ vcgencmd version
Mar 24 2022 13:19:26
Copyright (c) 2012 Broadcom
version e5a963efa66a1974127860b42e913d2374139ff5 (clean) (release) (start)

user@raspberrypi:~ $ dmesg | grep brcmf
[    6.086805] brcmfmac: F1 signature read @0x18000000=0x15264345
[    6.100425] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    6.101348] brcmfmac mmc1:0001:1: Direct firmware load for brcm/brcmfmac43455-sdio.raspberrypi,4-compute-module.bin failed with error -2
[    6.109713] usbcore: registered new interface driver brcmfmac
[    6.358819] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    6.358989] brcmfmac: brcmf_fw_alloc_request: using brcm/brcmfmac43455-sdio for chip BCM4345/6
[    6.368049] brcmfmac: brcmf_c_preinit_dcmds: Firmware: BCM4345/6 wl0: Nov  1 2021 00:37:25 version 7.45.241 (1a2f2fa CY) FWID 01-703fd60
[    7.214990] brcmfmac mmc1:0001:1 piwlan: renamed from wlan0
[    8.508320] brcmfmac: brcmf_cfg80211_set_power_mgmt: power save enabled

btmon.log dmesg.log

[pelwell]

Unfortunately we have no documentation of the parameters - the firmware and its configuration is a black box. The btc_mode=4 setting sounds interesting - I'm going to suggest it on the the other issue in case it helps.

[IB1387]

Hi @pelwell, thanks for your swift reply. Can you connect us with someone at infineon (I'd be glad to create an account there), who might be able to help finding the optimal set of parameters?

[pelwell]

I'm loathe to distract Infineon when it has taken so long to get anywhere on #12, but if you register on their system I can add you as a collaborator to the existing case.

[IB1387]

That works for me too! I've sent an e-mail with my details to the address that you posted in issue #12.

[onsolutionjames]

We recently updated the firmware, kernel, etc on our RPis and started having a similar issue, where suddenly BLE connections were failing 99% of the time. For us it turned out to be caused by the Wi-Fi powersave mode.

Wi-Fi powersave mode was completely disabled (through NetworkManager), and when we re-enabled it, our BLE issues immediately went away.

I'm not sure if it's the same issue as what you're having, but I just thought I'd mention it in case it helps.

[IB1387]

Hi @onsolutionjames, thanks a lot for sharing the solution that worked for you! Power_save was on for us by default. In a situation where we where actively experiencing the issue, turning off power_save using iw piwlan set power_save off did not help, though I'm not sure anymore if that setting is active immediately or needs a reboot. We might try this again to see if this improves things further.

We are still trying out btc_mode=4 with good success. It does not look like this is our only bluetooth issue (we sometimes also need to systemctl restart bluetooth because bluez seems to be stuck somehow), but the current situation is far better than before.

[pelwell]

Infineon have come back with the following:

1. Before starting the test (it can be connected to the AP already), please run this command: sudo wl -i wlan0 btc_params 51 0x40FF

2. When the issue occurs, report the output from these commands: sudo wl -i wlan0 btc_params 27 sudo wl -i wlan0 PM sudo wl -i wlan0 mpc

[IB1387]

Running btc_params 51 0x40FF while the issue occurs immediatly resolves the issue comparable to using btc_mode = 4. The wifi connection is slowed down after the command, also comparable to btc_mode = 4 but maybe even a little bit worse.

Running the other command while the issue occurs (with btc_mode = 1 and without running btc_params 51 0x40FF): ./wl64 -i piwlan btc_params 27 -> 0x16 (sometimes it also results in 0x17 or 0x15) ./wl64 -i piwlan PM -> 2 ./wl64 -i piwlan mpc -> 1

[pelwell]

@IB1387 I copied your previous answer verbatim to Infineon to avoid misunderstandings, but they have come back with a question:

regarding the feedback from another user - running btc_params 51 0x40FF resolving the issue. Are you saying that in this case the btc_params 51 does not change to 0x40bf?

Can you make clear the sequence of events, and any associated change to the value of btc_params 51?

[IB1387]

Sorry, I was out of office for a few days. I've redone the test to make sure the gathered data is valid.

Nearly all bluetooth connections fail while connected to a 2.4GHz wifi network using a completely stock RaspberryPi OS 64bit image (including btc_mode=1). Running ./wl btc_params 51 returns 0x190. As soon as I run ./wl btc_params 51 0x40ff bluetooth connections immediatly start to be established successfully. If I then set btc_params 51 to 0x190 again (using ./wl btc_params 51 0x190), bluetooth connections start to fail again. This seems to be repeatably from my testing.

I hope this answers the question from Infineon. Let me know if I can help any further.

[pelwell]

There's a new test firmware that sets btc parameter 51 to 0x40ff: https://drive.google.com/file/d/1pgQOt1X2vPK-6u2I1I7Pgp5DFasP3BYO/view?usp=share_link

[davidalo]

Following this issue, similar problem here. Using CYW4343W in custom board. I will test all of your proposals.

You can detailed see my problem here

[onsolutionjames]

I was just wondering if any progress has been made on this issue (or issue #12)? We've been having similar issues with BLE/Wi-Fi on the 3B+ and I suspect it's a coexistence issue. We have quite a lot of 3B+'s in the field (6000+) and we're happy to use some for testing.

[pelwell]

There was a firmware release from Infineon that seemed to solve the problem of one user/organisation by removing a priority boost given to scanning over Bluetooth activity; I don't know how @IB1387 got on with it.

You can download the new firmware to test here: https://drive.google.com/file/d/1cdH7Tpuuky8oJTaq2DlvyirLEoooYVkr/view?usp=sharing

Install it with:

$ sudo cp /lib/firmware/brcm/brcmfmac43430-sdio.bin{,.bak}
$ sudo cp brcmfmac43430-sdio_c2195fc6.bin /lib/firmware/brcm/brcmfmac43430-sdio.bin

and revert with:

$ sudo cp /lib/firmware/brcm/brcmfmac43430-sdio.bin{.bak,}

[sciguy14]

@pelwell that appears to be firmware for the CYW43430, but the Pi compute module uses the CYW43455 chipset, doesn't it?

[pelwell]

Ah yes - the sister issue (https://github.com/RPi-Distro/bluez-firmware/issues/12) is for 43430. It looks like we didn't have a similar case with Infineon for the 43455 so I've opened one. Hopefully it shouldn't take them long to change one value and rebuild.

[pelwell]

Infineon are holding out on simply giving us what we ask for and want us to run some diagnostics first. They've seen @IB1387's logs, but now say they want to see the values of some firmware parameters.

You'll need the wl utility, which can be downloaded from here: https://drive.google.com/file/d/1Q3bQl6T3LmyMSly_1yjUnVMRff_19tMG/view?usp=sharing

It's a 32-bit executable, but as a static build it should run on both 32-bit and 64-bit kernels and images. You may need to chmod +x wl to make it executable.

The values they would like to see are the outputs of these commands (assuming that wl is in the current directory):

$ sudo ./wl -i wlan0 btc_mode
$ sudo ./wl -i wlan0 btc_params 51

For comparison, I get the values 1 and 0x190, respectively.

They would like to know the values after booting and when you see connection issues - perhaps you could log the values every few seconds:

$ (while true; do
    sudo ./wl -i wlan0 btc_mode
    sudo ./wl -i wlan0 btc_params 51
    sleep 5
    done) | tee wl_log.txt

[pelwell]

You'll get nicer output with this version:

$ (while true; do
    echo $(date): $(sudo ./wl -i wlan0 btc_mode) $(sudo ./wl -i wlan0 btc_params 51)
    sleep 5
    done) | tee wl_log.txt

e.g.:

Thu 27 Jul 09:44:43 BST 2023: 1 0x190
Thu 27 Jul 09:44:48 BST 2023: 1 0x190
Thu 27 Jul 09:44:53 BST 2023: 1 0x190

[IB1387]

Hi everyone, sorry for the late answer, I just didn't monitor the activity here as closely anymore and some github mails went to spam. As I've already wrote here, our current and so far satisfactory solution is to just use btc_mode=4.

If you require additional info from my side, let me know.

[sciguy14]

@pelwell Mine doesn't change from 1 and 0x190 while trying and failing to pair a BLE devices:

Thu Jul 27 10:55:28 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:33 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:38 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:43 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:48 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:53 PM UTC 2023: 1 0x190
Thu Jul 27 10:55:58 PM UTC 2023: 1 0x190
Thu Jul 27 10:56:04 PM UTC 2023: 1 0x190
Thu Jul 27 10:56:09 PM UTC 2023: 1 0x190
Thu Jul 27 10:56:14 PM UTC 2023: 1 0x190
Thu Jul 27 10:56:19 PM UTC 2023: 1 0x190

I did try just forcibly setting btc_mode to 4 and/or setting btc_params51 to 0x40ff in the NVRAM, and when I do that, the above script obviously just returns the new value I have set it to via the NVRAM. However, BLE pairing fails with equal frequency in either case - changing these values doesn't appear to help anything for me.

[pelwell]

Your findings have been passed on.

[pelwell]

After some back-and-forth Infineon have responded with the following:

The disconnection always happens with:

> HCI Event: LE Meta Event (0x3e) plen 12                  #529 [hci0] 6.509780
      LE Read Remote Used Features (0x04)
        Status: Connection Failed to be Established (0x3e)
        Handle: 64
        Features: 0x3f 0x00 0x00 0x08 0x00 0x00 0x00 0x00

What are you using as the peer device?  Why it supports so few feature?

Could you do the following test? (not at the same time)
1. make the timeout longer. 4s.
2. after you send ' LE Read Remote Used Features' , add 2s delay before the next command(remove from whitelist).
3. try to set btc_params 51 0x1c0
4. What are values of btc_params 1, 9, 8, 10  ?

[davidalo]

There was a firmware release from Infineon that seemed to solve the problem of one user/organisation by removing a priority boost given to scanning over Bluetooth activity; I don't know how @IB1387 got on with it.

You can download the new firmware to test here: https://drive.google.com/file/d/1cdH7Tpuuky8oJTaq2DlvyirLEoooYVkr/view?usp=sharing

Install it with:

$ sudo cp /lib/firmware/brcm/brcmfmac43430-sdio.bin{,.bak}
$ sudo cp brcmfmac43430-sdio_c2195fc6.bin /lib/firmware/brcm/brcmfmac43430-sdio.bin

and revert with:

$ sudo cp /lib/firmware/brcm/brcmfmac43430-sdio.bin{.bak,}

This firmware is fixing the issue for me. Could anyone offer more details about where did this firmware come from? Version? Is it official?

[pelwell]

It's a test firmware they built for us. If we get enough positive feedback it could become the official release (or they will make a release with the same change).