BTHome (Passive Scanning) does not detect new devices and does not work with already registered ones.

[pvvx]

Describe the bug

If in 'Bluetooth' integration enabled "Passive Scanning": 'BTHome' does not detect new devices and does not work with already registered.

To Reproduce

Config: Home Assistant 2022.11.1, Bluez 5.65 (enable 'experimental') Two USB-BT adapters. One is connected to 'Passive BLE Monitor' integration, the other is connected to 'Bluetooth'.

Additional context

'Qingping' and 'Xiaomi BLE' integrations work successfully.

If duplicate the sensor from 'Qingping' to 'BTHome', then it works in 'BTHome'.

If disable "Passive Scanning" in 'Bluetooth', then everything in 'BTHome' starts working. But there is a constant overwriting of the bluetooth cache files, and reduces the battery life in the sensors.

[pvvx]

Found this behavior: If there is no "BLE Advertising Data Flags Field" (02, 01, BLE_GAP_ADV_FLAGS) in the advertisement package, then "BTHome" does not work with Passive Scanning.

PS: "BLE Advertising Data Flags" is ignored by Android and iOS. With any flag bits, Android and iOS allows you to connect to these devices. For this reason, today it is superfluous and its entire purpose is to additionally consume the battery of the device.

---

Addition:

If 'Active Scanning' is disabled and the device has flag: ADTYPE_LE_LIMITED_DISCOVERABLE_MODE then 'BTHome' does not work with that device. If Active Scanning is enabled, then the flags are unimportant.

I wonder what the logic is in this?

[Ernst79]

That is an interesting finding. So we should advise to include (02, 01, BLE_GAP_ADV_FLAGS) in each advertisement. I remember Xiaomi uses 020106 in most advertisements, but not all.

This is one for example (of a Xiaomi sensor) that doesn't have a 0201xx data flag field.

043e22020100004c94b438c1a416151695fe50305b05034c94b438c1a40d10041001ea01cf

[Ernst79]

So (for my understanding), Xiaomi is using LE General Discoverable Mode + BR/EDR not supported. (02 + 04 = 06)

• BR/EDR not supported. --> BLE mode (not Bluetooth Classic)
• LE General Discoverable Mode

Discoverability modes

• Non-discoverable mode — Other devices that are scanning cannot detect the presence of a peripheral in non-discoverable mode. A peripheral uses this mode to not get discovered by and connected to a peer device. For the same effect, the peripheral can advertise with the flags ADV_NONCONN_IND or ADV_SCAN_IND set in the Flag AD field in the advertising packet.
• Limited discoverable mode — The Limited Discoverable Flag is set in the Flag AD field of the advertising packet and the device is discoverable for a limited time with lower priority. A Peripheral device in limited discoverable mode can be discovered only by a Central device that is following the limited discovery procedure.
• General discoverable mode — In this mode, a Peripheral device can be seen by a Central device as long as it is advertising. Generally, the intention is to get discovered and establish a connection, so the General Discoverable Flag is set in the Flag AD field of the advertising packet.

/**@defgroup BLE_GAP_ADV_FLAGS GAP Advertisement Flags
 * @{ */
#define BLE_GAP_ADV_FLAG_LE_LIMITED_DISC_MODE         (0x01)   /**< LE Limited Discoverable Mode. */
#define BLE_GAP_ADV_FLAG_LE_GENERAL_DISC_MODE         (0x02)   /**< LE General Discoverable Mode. */
#define BLE_GAP_ADV_FLAG_BR_EDR_NOT_SUPPORTED         (0x04)   /**< BR/EDR not supported. */
#define BLE_GAP_ADV_FLAG_LE_BR_EDR_CONTROLLER         (0x08)   /**< Simultaneous LE and BR/EDR, Controller. */
#define BLE_GAP_ADV_FLAG_LE_BR_EDR_HOST               (0x10)   /**< Simultaneous LE and BR/EDR, Host. */

[pvvx]

TelinkMiFlasher has long provided the possibility of including flags in the advertising package: But this is all a temporary solution. Disambiguation is required when working with active and passive scanning. With over 30 BLE devices, I can't use active scanning on the current HA implementation with 'Bluetooth' on a standalone system - too many SSD writes. 'HA' on small system with MMC:

[Ernst79]

So, if I understand you right, you prefer, in BTHome mode, to not send a flags packet (020106). And HA Bluetooth should, without this flags packet, still work in passive mode, which it currently does not.

Perhaps @bdraco knows why this isn’t working

PS In qingping and Xiaomi mode, is the flags packet always included?

[pvvx]

In qingping and Xiaomi mode, is the flags packet always included?

Yes.

But in active scan mode (Bluez), the flags are not considered.

---

PS: There are still ambiguities and the lack of a description for working in Coded PHY (Long Range). Bluez with BT5.1+ adapters does not accept some extended advertising combinations in Coded PHY. But even most smartphones do not understand extended advertising if Coded PHY is used on the main advertising channels. This completely eliminates the use of long-distance communication in BLE. But an alternative is possible. Transmission of the label "extended advertising" on PHY 1M, and the body itself on a fixed channel in Coded PHY. Even Bluez understands this combination...

[Ernst79]

I have asked @bdraco about this issue, he says that not processing data without the flags packet in passive mode is caused by bluez, as it is hardcoded in bluez. BLE monitor does not use bluez, that is probably why BLE monitor is accepting packets without flags, while the new Bluetooth integration in HA, which uses bluez, isn't.

You'll have to send the flags packet or bluez will not accept it with the Advertisement Monitor (passive). I'm pretty sure it will accept just about anything with active scans though We can't change that behavior as its hard coded into bluez

So, not easy to fix, I'm afraid. I'll add a note in the documentation Added a note in the documentation.

[catch-twenty-two]

If disable "Passive Scanning" in 'Bluetooth', then everything in 'BTHome' starts working. But there is a constant overwriting of the bluetooth cache files, and reduces the battery life in the sensors.

Seeing the same problem, constant churning on my IO 24/7 by bluetoothd unless bluetooth is disabled in home assistant. It wasn't clear to me how you ended up fixing this? Or did you?

[catch-twenty-two]

For others that are worried about SSD/HD wear, as a temporary fix I ended up auto mounting a ram disk at /var/lib/bluetooth/ no more churn, but really just a bandaid.

[pvvx]

This is specific to Bluez and cannot be fixed in Home Assistant «Bluetooth» while Bluez is in use.

Switch to the passive scanning mode, and on the thermometers, enable the transmission of Flags.

Additionally, if the "active scanning" mode is enabled, the BT adapter receives fewer BLE advertisements, i.e. there will be more missed values and events from sensors.

[50494554524F]

For others that are worried about SSD/HD wear, as a temporary fix I ended up auto mounting a ram disk at /var/lib/bluetooth/ no more churn, but really just a bandaid.

can you tell us how to do that?!

thanks

[Ernst79]

Closing this issue. The question in the opening post can be solved by enabling AdvFlags in the pvvx firmware.