pvvx / ATC_MiThermometer

Custom firmware for the Xiaomi Thermometers and Telink Flasher
https://github.com/pvvx/pvvx.github.io/tree/master/ATC_MiThermometer
Other
2.75k stars 196 forks source link

Name of the device not show in Home Assistant #426

Closed Misiu closed 6 months ago

Misiu commented 7 months ago

First of all, thank you for the project, I've just flashed my first LYWSD03MMC, the process was so easy and well-described. After uploading firmware 4.5 and setting the advertising type to BTHome v2 (and saving config) the device showed up in Home Assistant as:

ha_discovery

I've set the device name and that name is shown when I try to connect to a Bluetooth device from my browser and when I use Get device Name, but it isn't used in Home Assistant.

image

Before I start an issue in the Home Assistant repo I'd like to be sure that the Local Name is sent in the advertising payload (according to the format: https://bthome.io/format/)

I'm probably not the only one who plans to use more than one device, so adding a custom name and seeing that name in HA would be very helpful.

pvvx commented 7 months ago

Your image displays the name it received when you scanned it.

The BTHome format does not contain the device name. There are standard Bluetooth identifiers (0x09 - Complete local name) for this in BT/BLE. The device name is usually sent when requesting an active scan.

Additional information from the device can be obtained by requesting standard UUIDs.

Misiu commented 7 months ago

@pvvx thank you for your reply. If I understand correctly we can't set a complete local name based on the device name, is this right? I'm not sure what and where should be changed to get the correct behavior - the device name should be shown in HA in discovered devices. I use Shelly as BT Proxy, it has the passive scan selected. I've changed it to active, but this didn't change.

If everything is correct on your site (I just wanted to be sure) I'll create an issue in HA Repo. To be 100% sure, the device name we can set using Flasher isn't related to the advertising type and is a standard BT feature?

pvvx commented 7 months ago

Yes. Both in the BT advertising message and in the attributes.

Home Assistant does not read this information. But they promised... a long time ago...

pvvx commented 7 months ago

There is a lot of useful stuff missing from Home Assistant. I can't move this matter forward alone. The only responses I receive are references to using Bluez or other APIs, which prevent them from working. And following the chain where they direct you, you will get to the Linux kernel. But it has its own cockroaches and there has been no full support for BT4.2 since 2016.

And now Bluetooth version 5.4. Android supports Bloetooth version 5+...

Misiu commented 7 months ago

@pvvx I've created a new issue, let's wait for the devs' response. The device name is set here: https://github.com/Bluetooth-Devices/bthome-ble/blob/4cb68589dabe995df04bb6abf2cbc9a1e62d9687/src/bthome_ble/parser.py#L266 I'll try to set up the environment and debug it.

Sadly I'm not sure how this works when using ESPHome or a Shelly based proxy.

doe81 commented 7 months ago

image I have several named devices. So it works.

Misiu commented 7 months ago

I've tried that. I've flashed the device, set the name, restarted HA and then checked for discovered devices.

Do you have BT on the machine running HA or are you using BT Proxy?

doe81 commented 7 months ago

Do you have BT on the machine running HA or are you using BT Proxy?

I'm running Ubuntu and HA in Docker. Im using the built-in BT.

Misiu commented 7 months ago

I'm running Ubuntu and HA in Docker. I'm using the built-in BT.

That might be the reason. I'm running HAOS on Dell 5070 and I'm using Shelly Gen 2 devices as BT Proxy.

doe81 commented 7 months ago

I'm running HAOS on Dell 5070 If it has an builtin BT try using it or connect a external BT device. Cloud be the proxy feature not sending all info. Try to add it first with built-in and then use the proxy.

Misiu commented 7 months ago

I'll check if I have a BT adapter I could use, but ideally, this should work on BT Proxy and BT adapters that are directly connected to the device running HA.

pvvx commented 7 months ago

I'm running Ubuntu and HA in Docker. Im using the built-in BT.

Do you have active scanning enabled?

image image

Misiu commented 7 months ago

@pvvx I don't have Bluetooth integration. As mentioned before I use ESPHome Bluetooth Proxy and Shelly Gen 2 devices, all of them has active scanning set: image

pvvx commented 7 months ago

It was a question for @doe81.

The device only transmits the name to an active scan request (In Legacy mode (BT4.2)). The name can be accepted by the adapter if there is another actively scanning device on the airwaves. And also, when the thermometer is included in the function of the Le Long Range mode - "Extended Advertisements". Then the length of the advertising message is not limited and the name is always transmitted. Such implementation is used in current (and past) firmwares.

pvvx commented 7 months ago
  1. Renamed the device (TelinkMiFlasher).
  2. Rebooted everything (НА).
  3. BTHome options in HA: image image image
  4. Then rebooted everything again.
  5. Waited for more than 15 minutes. The device does not appear in the BTHome list.
  6. Did everything (1..5) again 3 times. The device does not appear in the BTHome list.
  7. Switched the adapter to accept all PHY formats (2M/1M/Coded).
  8. Repeated steps 1..6. The device does not appear in the BTHome list.

No docker or other rubbish - everything is in pure Linux, and under “root” rights.

The RTL8761BU adapter is connected to the BTHome. CSR8510A10 adapter is connected to the "Passive BLE monitor".

image

The renamed device continues to work successfully in "Ble Passive Monitor":

image

Several other BLE devices work on the RTL8761BU adapter with BTHome, but the new one is not recognized.


  1. Changed the BLE advertising interval to 2 seconds. The device does not appear in the BTHome list.
  2. Checking settings: /etc/bluetooth/main.conf
    # LE scanning parameters used for passive scanning supporting auto connect
    # scenarios
    #ScanIntervalAutoConnect=
    #ScanWindowAutoConnect=
    ScanIntervalAutoConnect=22000

    /lib/systemd/system/bluetooth.service

    ExecStart=/usr/libexec/bluetooth/bluetoothd --experimental

    Everything is fine. But the device is not displayed in BTHome.


The system is used exclusively for Home Assistant. For debugging thermometers. No other applications are installed.

  1. reboot

  2. I wait 10 minutes.

And the device operating in the BTHome ver2 format is not one, but not a single new BTHome integration appears. At the same time, these devices work well in "Passive Ble Monitor".

  1. system # poweroff and power off UPS. Restart...
  2. I wait 10 minutes.

Nothing helps the BTHome integration. Old, long-recognized devices continue to work, but new ones are not recognized.

What's wrong with BTHome?


  1. I switch the renamed thermometer to the "MiHome" format.
  2. After a couple of seconds in HA: image

This means that the adapter is not at fault. Something in BTHome.

pvvx commented 7 months ago

The source of the glitches has been found. But the reason is unknown. The nonsense carries "Bluetooth" integration:

2023-12-01 21:01:05.487 DEBUG (MainThread) [homeassistant.components.bluetooth.manager] hci1 (8C:88:2B:00:E7:EA) [connectable]: 58:2D:34:14:50:87 AdvertisementData(local_name='CGG_145087', service_data={'0000fcd2-0000-1000-8000-00805f9b34fb': b'@\x00\xfc\x01d\x02]\x08\x03\x1f\x11', '0000fe95-0000-1000-8000-00805f9b34fb': b'PXH\x0b\xd6\x87P\x144-X\r\x10\x04\xd8\x00\xa9\x01'}, rssi=-73) match: set()

Some kind of mixture of MiHome and BTHome.

But in reality the device transmits the following: image

That is, everything is normal and clearly in the BTHome v2 format.

btmon -i 1

> HCI Event: LE Meta Event (0x3e) plen 44
      LE Extended Advertising Report (0x0d)
        Num reports: 1
        Entry 0
          Event type: 0x0013
            Props: 0x0013
              Connectable
              Scannable
              Use legacy advertising PDUs
            Data status: Complete
          Legacy PDU Type: ADV_IND (0x0013)
          Address type: Public (0x00)
          Address: 58:2D:34:14:50:87 (Qingping Electronics (Suzhou) Co., Ltd)
          Primary PHY: LE 1M
          Secondary PHY: No packets
          SID: no ADI field (0xff)
          TX power: 127 dBm
          RSSI: -75 dBm (0xb5)
          Periodic advertising interval: 0.00 msec (0x0000)
          Direct address type: Public (0x00)
          Direct address: 00:00:00:00:00:00 (OUI 00-00-00)
          Data length: 0x12
        02 01 06 0e 16 d2 fc 40 00 e4 01 64 02 67 08 03  .......@...d.g..
        a5 10                                            ..
        Flags: 0x06
          LE General Discoverable Mode
          BR/EDR Not Supported
        Service Data: Unknown (0xfcd2)
          Data: 4000e4016402670803a510
@ MGMT Event: ADV Monitor Device Found (0x002f) plen 34
        Handle: 0
        Address: 58:2D:34:14:50:87 (Qingping Electronics (Suzhou) Co., Ltd)
        Addr Type: 1
        RSSI: -75
        Flags: 0x00000000
        AD Data Len: 18
        AD Data: 0201060e16d2fc4000e4016402670803a510

And it’s not surprising that somewhere a name is written down, but somewhere not :)

Misiu commented 7 months ago

So this should be reporter in HA Core? Or in BTHome repo?

Not sure if this is related, but I have BT Home and Xiaomi BLE integrations in my HA instance.

pvvx commented 7 months ago

Already.

I wanted to check the device name update in BTHome, but nothing happened :) In general, as always with Bluetooth integration based on Bluez.

pvvx commented 7 months ago

So this should be reporter in HA Core? Or in BTHome repo?

Bluez caches names and stuff. But HA doesn’t know about this or hides it, because they chose the most buggy option - Bluez.

https://stackoverflow.com/questions/43446918/bluez-showing-old-cached-data-on-dbus https://github.com/home-assistant/operating-system/commit/f8f2e61967eab33209531e932e7e98b3c8efc1a7

pvvx commented 7 months ago

I'll check if I have a BT adapter I could use, but ideally, this should work on BT Proxy and BT adapters that are directly connected to the device running HA.

If the proxy does not report the device name, how will it reach the HA?

I have several named devices. So it works.

  • if you rename the device after it is detected restart HA and it will detect the new name.
  • if you rename the device after it is configured remove the device and restart HA and it will detect the new name

This doesn't work 90% of the time. BLE devices transmit the name upon active scanning request. But, if active scanning is enabled, then the batteries of all your BLE devices will drain much faster (by 25%) from the additional transmission each advertising period. For this reason, gateways and many users use “passive scanning”. For example, the Xiaomi 3 gateway constantly operates in passive scanning mode, and when a new device appears, it polls it once with an active request. This is not implemented in Bluez and HA.

doe81 commented 7 months ago

I'm running Ubuntu and HA in Docker. Im using the built-in BT.

Do you have active scanning enabled?

image image

@pvvx Yes, I had active scanning enabled. Should I select passive scanning instead?

pvvx commented 7 months ago

https://community.home-assistant.io/t/passive-ble-monitor-integration/303583/586 https://www.google.com/search?q=ble+active+vs+passive+scanning

After each transmission of a BLE advertising packet, the device waits (512 μs) for a request to transmit additional information or a connection request. During passive scanning, the BT ver 4.2 device transmits 3 packets (on 3 channels) every advertising interval. During an active scan, the device receives a request and transmits an additional packet of information. As a result, we get 4 transmissions per BLE advertising interval, plus the energy for the time it takes to receive and process the request packet. As a result, the battery discharge increases to 25%.

The relationship is nonlinear because CR2032 does not like pulsed load. The results of increased consumption may be greater.

The reasons why periodic active polling of BLE devices is not done in HA is unknown. The same reasons lie in the use of Bluez and other outdated software that does not allow users to work in modern BLE, WiFi, and Zigbee standards.