search

custom-components / ble_monitor

BLE monitor for passive BLE sensors
https://community.home-assistant.io/t/passive-ble-monitor-integration/
MIT License
1.9k stars 247 forks source link

[HA]: Parse data from ESPHome and HA 2022.08+ #978

Closed andrewjswan closed 2 years ago

andrewjswan commented 2 years ago

What happened?

Since BLE Monitor will soon move to Deprecated status, I would like to understand the fate of Parse data from ESPHome.

At the moment in HA 2022.08 version service ble_monitor.parse_data does not exist, as well as there are no Mac addresses in the attributes (But Mac addresses are needed to send them to the gateway) of entities.

Question, is it planned to transfer this functionality to HA?

  1. Service: ble_monitor.parse_data
  2. Sensor attributes (without saving them to the recorder) such as:
    • Sensor type - LYWSD03MMC
    • MAC address - A4:C1:XX:XX:XX:XX
    • Firmware - Xiaomi (MiBeacon V5 encrypted)

And the question about entity_id, they can not be adjusted according to the old name? Only to rename them by hand?

Sensor type

All

Relevant log output

No response

chertvl commented 2 years ago

+1, necessary thing

Ernst79 commented 2 years ago

There will be a possibility to forward BLE data from ESPhome to HA Bluetooth integration. If this data is supported by one of the new integrations, it will be picked up and parsed. But this will be in a future release, till that time BLE monitor will keep working. Balloob himself was very keen on having this in ESPhome and HA, so I have no doubt this will be created in the near future. But please be patient, they/we cant convert BLE monitor to core integrations in one month time.

About your second question, there is already created an issue to add the Mac address to the device info (not entity attributes). But please make a request/issue on the HA core github for this, as the devs wont look here for these ideas.

The naming is following some rules in HA now, so they cant make the exact same name as in BLE monitor unfortunately. That means renaming by hand for automatisch and lovelace unfortunately, sorry.

andrewjswan commented 2 years ago

https://github.com/home-assistant/core/issues/76163

andrewjswan commented 2 years ago

@Ernst79 Have you heard anything about the new functionality in ESPHome that this link talks about?

https://github.com/home-assistant/core/issues/76163#issuecomment-1212966556

Jc2k commented 2 years ago

The functionality I talk about in that link is the same that @Ernst79 is talking about just with a different perspective; we were both present in the chats about it.

But as @Ernst79 says, please be patient. This is all very new, the Xiaomi integration isn't even a month old, the bluetooth integration isn't much older. It's going to take a few releases to shake things out. You don't have to rush to change your setup yet.

Ernst79 commented 2 years ago

@andrewjswan see the response of @Jc2k. I have no doubt that it will come, but it takes time. There are more things to do, the devs cant do it all at once. So please be patient. Till that time, continue to use BLE monitor.

Ernst79 commented 2 years ago

I think this issue can be closed. You can use ESPHome now to forward all data to the new BLE integrations with https://esphome.github.io/bluetooth-proxies/

Keep an eye on tonight's release presentation, there will most likely be some more information about this new feature.

Other issues related to the new integrations should be raised at the home assistant core github page, not here.

andrewjswan commented 2 years ago

@Ernst79 Do you know if this proxy accepts everything or only MAC addresses registered in HA?

Ernst79 commented 2 years ago

If I remember correct, it just forwards everything, and in HA you can choose what to enable and what not (in the integrations). But they were also working on implementing some filters to reduce the load on the ESP. But I think that isn’t available yet. They were experimenting with RSSI filtering, MAC whitelists and Mac blacklists. Not what will make it to the final version of ESPHome. Perhaps you can ask tonight on the release party of HA 2022. @bdraco will be there too, he can probably explain it better than me.

Jc2k commented 2 years ago

From what I understand, it has been tested in a couple of highly dense BLE environments without any problems too and the testing revealed that Home Assistant struggled more than the ESP did (it turns out we aren't using the "optimised" protobuf code path, so getting a ~80% speed boost will be quite easy, and hopefully in 2022.10).

It's also worth mentioning there's support for multiple USB dongles - include "remote" dongles using usbip.

andrewjswan commented 2 years ago

Here I am more worried not for Esp, but for Home Assistant, I now use two BLE gateways, and in Home Assistant I get about 4000 packets per hour, this is with filtering by Mac, if I turn off filtering by Mac, then packets are much more and the CPU load on Nuc (i3), reaches 50-60% although in the normal state only 4%.

image

Jc2k commented 2 years ago

As I said above, you need to use the fast version of protobuf. Unfortunately the default seems to be the slow one. HA is planning to make the default the fast one in its containers, unfortunately that didn't make it into 2022.10.

andrewjswan commented 2 years ago

I don't plan on switching to HA Bluetooth yet, not all devices are supported, and the integration itself constantly scans the airwaves. And this, I understand I can not disable. I prefer to add the necessary devices by hand, rather than clean out all of the neighbors added through the auto search. So I'll wait for the news. Maybe in 2022.11 something will change.

andrewjswan commented 2 years ago

Will Discovery and constant scanning be able to be turned off (To avoid catching all 100 neighborhood sensors)? And will it be possible to add your sensors manually, select integration, enter the MAC address and add?

IronButterfly commented 2 years ago

Just turned on the BLE integration and added the found Xiaomi Mi Flora. The result is not happy at all.

Screenshot 2022-09-09 at 15 23 52
Jc2k commented 2 years ago

As I said above, this is because your HA is using a python implementation of protobuf, rather than the c++ version.

The fix is known but it needs integrating into the container build process. The guy that can do this wont be available until the end of the month.

Jc2k commented 2 years ago

I'm not aware of any plans to support adding by MAC address.

Jc2k commented 2 years ago

Passive mode is supported by HA already. But it requires your host BlueZ to be set up correctly. Unfortunately that means turning on experimental mode, it is not the default on any OS AFAIK.

andrewjswan commented 2 years ago

@Jc2k What I meant was, we added all the devices, and turned off the Discovery, why do we need neighborhood sensors? Especially when there are a lot of them, they clog up the entire setup page. It is convenient when there is nothing unnecessary. And very convenient when you bought a device, added to the MyHome, took the MAC and added to the HA hands. No searching, etc.

Jc2k commented 2 years ago

That sounds like a discussion that shouldn't happen here, and it's Not a question I can answer. If it's something you are willing to help implement you could start an arch issue. Otherwise it's probably best to start a feature request in the forums.

andrewjswan commented 2 years ago

Future request on the forum is a cemetery. :( There are a lot of them, and hardly any realized ones. :(

Jc2k commented 2 years ago

Unfortunately HA is asymmetrical - lots more users and ideas than developers or time. And most of the developers are specialised on one or 2 integrations. Not really much we can do :/