home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70.17k stars 29.2k forks source link

iBeacons without names are not seen by the integration #80357

Closed Sorien closed 6 months ago

Sorien commented 1 year ago

The problem

im testing iBeacon with android app https://play.google.com/store/apps/details?id=net.alea.beaconsimulator&hl=en&gl=US iBeacom is detected and integreation configured but no device added

image

What version of Home Assistant Core has the issue?

2022.10.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant Container

Integration causing the issue

iBeacon Tracker

Link to integration documentation on our website

No response

Diagnostics information

iBeacon packets are transmitted and looks ok verified by Acrylic Bluetooth LE Analyzer

image

but no device visible in home assistant

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

wastez commented 1 year ago

As far as i know chipolo one spot is no iBeacon devices, it works on the same principe as AirTag as I guess. Just chipolo one is a iBeacon device.

J4gQBqqR commented 1 year ago

Same here. Using docker container in Arch Linux. Bluetooth LE tracker is detecting a whole lot of devices MACs -> This means my bluetooth inside of my container is working.

I have three 24*7 powered android companion app with iBeacon monitor turned on. These monitors are shown as entities in my HA server. And on their Android companion app UI, I can see iBeacon detected. -> This means these companion app monitor sensor can detect iBeacon UUID.

Then in the server, I installed the iBeacon integration. Nothing shows up. -> This means what Android companion app detected is not passed into the HA server's iBeacon tracker integration correctly.

图片

图片

图片

wastez commented 1 year ago

If you are using chipolo one and you want to use it as iBeacon you must not pair it with the app to your phone. If you already have connect it with your phone you have to delete it, only then it will work.

Edit: Chipolo one spot will not work at all.

Auroray commented 1 year ago

Same here. Running HAOS on a RPi 4.

Tried:

  1. On-board BT passive / active = No iBeacon devices detected
  2. External BT adapter - Asus BT500, listed as officially supported = No iBeacon devices detected
  3. BT proxy - Shelly 1 Plus with latest firmware, all BT checkboxes enabled = No iBeacon devices detected
gtbdf1 commented 1 year ago

Same issue here. iBeacon not picking up any devices. It worked about 6 months ago but is no longer. Just says no devices.

dphuk commented 1 year ago

Bought 3 of these: https://www.aliexpress.com/item/1005003211033416.html. nRF52810 is on the supported list but on my NUC running HA I can't seem to discover anything new. I did have some iBeacon devices listed from last year, not sure what they even were, so it had been detecting items at some point. Other Bluetooth SwitchBot items are all working, so not quite sure how to troubleshoot this? [Home Assistant 2023.7.1, Supervisor 2023.07.1, Operating System 10.3]

wastez commented 1 year ago

@gtbdf1 @dphuk Does your device have a name? Otherwise it will not be detected if the iBeacon device has no name until you change the code. But if you have a iBeacon device from china you normally can add s name via a software.

With my chipolo I don't had the options to add a device name, so I had to edit the code, but then it will work.

dphuk commented 1 year ago

Using BLE Scanner it seems to show as "Holy-IOT"

@gtbdf1 @dphuk Does your device have a name? Otherwise it will not be detected if the iBeacon device has no name until you change the code. But if you have a iBeacon device from china you normally can add s name via a software.

With my chipolo I don't had the options to add a device name, so I had to edit the code, but then it will work.

Oooh, thank you for pointing me in the right direction. So it had a name of "Holy-IOT", I endedup getting the app "holyiot-beacon" then connecting to the device using the default password "aa14061112", from there I could change the type to "iBeacon" however this seemed to wipe out the UUID and Minor numbers, so I created those and now I have 4 devices showing, which is odd as only one has a battery in, but I'm getting somewhere now, will update later on, thank you!

dphuk commented 1 year ago

Using BLE Scanner it seems to show as "Holy-IOT"

@gtbdf1 @dphuk Does your device have a name? Otherwise it will not be detected if the iBeacon device has no name until you change the code. But if you have a iBeacon device from china you normally can add s name via a software. With my chipolo I don't had the options to add a device name, so I had to edit the code, but then it will work.

Oooh, thank you for pointing me in the right direction. So it had a name of "Holy-IOT", I endedup getting the app "holyiot-beacon" then connecting to the device using the default password "aa14061112", from there I could change the type to "iBeacon" however this seemed to wipe out the UUID and Minor numbers, so I created those and now I have 4 devices showing, which is odd as only one has a battery in, but I'm getting somewhere now, will update later on, thank you!

All seems to be working, thank you. The tag is now (very firmly) stuck to the bin out of sight. time will tell what the battery life is on this, but the automation is now updated to check if the bins still need to be moved out before alerting me :)
image

Fonsohome commented 1 year ago

@gtbdf1 @dphuk ¿Tu dispositivo tiene un nombre? De lo contrario, no se detectará si el dispositivo iBeacon no tiene nombre hasta que cambie el código. Pero si tiene un dispositivo iBeacon de China, normalmente puede agregar el nombre a través de un software.

Con mi chipolo no tenía las opciones para agregar un nombre de dispositivo, así que tuve que editar el código, pero luego funcionará.

how did you edit the chipolo code?

wastez commented 1 year ago

@Fonsohome The code of iBeacon in home assistant has to be edited so that iBeacons without name aren't ignored. If you need it i can tell you which lines you have to edit in the iBeacon code.

gtbdf1 commented 1 year ago

Yes, please do.

On Fri, Jul 14, 2023 at 9:17 AM wastez @.***> wrote:

@Fonsohome https://github.com/Fonsohome The code of iBeacon in home assistant has to be edited so that iBeacons without name aren't ignored. If you need it i can tell you which lines you have to edit in the iBeacon code.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/80357#issuecomment-1635853414, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALT2OEN2YVGWYA7XHPMUNIDXQFBGHANCNFSM6AAAAAARFSW2MQ . You are receiving this because you were mentioned.Message ID: @.***>

wastez commented 1 year ago

You have to go into the docker container from home-assistant. (the easiest way to do this is via the advanced ssh addon)

Then you have to edit the following file: /usr/src/homeassistant/homeassistant/components/ibeacon/coordinator.py

There you have to comment this part of the code that it will look like this:

        # Reject creating new trackers if the name is not set
#        if new and (
#            service_info.device.name is None
#            or service_info.device.name.replace("-", ":") == service_info.device.address
#        ):
#            return

After editing restart the machine and thats it, now home assistant will also detect not named devices like chipolo. The only bad thing is that it have to redone after every core update. (totaly clear because the docker container will be replaced)

I already asked the coders of it to remove this but they don't want to do it.

leeuwte commented 1 year ago

You can always copy the code and make it a custom integration with those changes.

wastez commented 1 year ago

You can always copy the code and make it a custom integration with those changes.

So the custom integration will overrule the default one?

leeuwte commented 1 year ago

yes it will, you only need to add a version property in the manifest.

Fonsohome commented 11 months ago

You can always copy the code and make it a custom integration with those changes.

So the custom integration will overrule the default one?

I would appreciate it because I can't get to the file by ssh the path ends in src.

Misiu commented 10 months ago

I have the same problem (empty iBeacon Tracker). I have the latest HA App (2023.9.2) installed on my phone (Huawei P20), Bluetooth enabled on my phone, and BLE transmitter sensor enabled. I have 3 Shelly devices with BLE gateway enabled and ESP Home with those settings:

esp32_ble_tracker:
  scan_parameters:
    active: false

bluetooth_proxy:
  active: false

As soon as I enabled BLE transmitter sensor in app I saw iBeacon Tracker integration, I added it, but I see no devices (trackers) in it. What should I do? Is this a bug in iBeacon Tracker?

mhetzi commented 10 months ago

Why was this added, when the companion app does not send names? Wondered why my new phone didnt show up in ibeacon tracker, but the old one did...

Could we have this toggable in the config flow?

dshokouhi commented 9 months ago

its working here, latest android app and latest HA core. Keep in mind the default settings may not work for your own particular setup. You need to adjust sensor settings. Not all android devices will behave the same with same settings.

image

image

mhetzi commented 9 months ago

After removing the namecheck it works really well Btw: this problem even existed by simply upgrading the android version. so one does not have to buy a new device.

image image

Room Detection works great to: image

bdraco commented 9 months ago

The integration is explicitly not supporting iBeacon's without a name set because it created too many transient entities during beta testing.

For users with many nameless iBeacons around them this will eventually slow down HA and become a problem so we can't enable support for ones without a name by default without handing some users a bad day in the future when the entity count gets so high that their instance falls over

mhetzi commented 9 months ago

Would whitlisting the uuid be a problem, its not autodetecting but better than not working?

dshokouhi commented 9 months ago

@bdraco what about user provided UUID? Those can be set in the android app so it might be nice to give users that access. Maybe a special white list? The issue is that every android device is going to be different here, even our default settings don't work for all devices. In the case of the android app we have a lot more control than other devices so I think it makes sense if its doable. If any changes are needed in the android app let us know and we can work on it.

bdraco commented 9 months ago

If someone wants to pick up being another code owner and add more features to the integration that would be great. I don't have the bandwidth to do anything more than fix bugs (and code review) for it currently.

For clarity: I'm not opposed to name support being an option (as long as its off by default), but that comes with someone also picking up code owner to deal with any fallout from that change since I won't be able to do that.

Misiu commented 9 months ago

its working here, latest android app and latest HA core. Keep in mind the default settings may not work for your own particular setup. You need to adjust sensor settings. Not all android devices will behave the same with same settings.

I have Huawei P20 (running Android 10), Home Assistant App in version 2029.9.2-full and Home Assistant in version 2023.10.1. As soon as I enabled BLE transmitter sensor in the app, I got iBeacon Tracker integration discovery notice. I'm not sure what else do I must enable/change in my phone/app to get tracking working.

The integration is explicitly not supporting iBeacon's without a name set because it created too many transient entities during beta testing. As mentioned here this might be my phone's fault, because it may not send the name.

So list of allowed UUIDs would be the best option.

Sorien commented 9 months ago

@bdraco what about user provided UUID? Those can be set in the android app so it might be nice to give users that access. Maybe a special white list? The issue is that every android device is going to be different here, even our default settings don't work for all devices. In the case of the android app we have a lot more control than other devices so I think it makes sense if its doable. If any changes are needed in the android app let us know and we can work on it.

@dshokouhi copied from my own app, home assistant android app just needs to handle scan response to send device name

 AdvertiseData scanResponse = new AdvertiseData.Builder()
                    .setIncludeDeviceName(true)
                    .build();

mBluetoothLeAdvertiser.startAdvertisingSet(parameters, dataBuilder.build(), scanResponse, null, null, mAdvertiseCallback);
mhetzi commented 9 months ago

@bdraco I tried something, and it works in my case. For testing in my instance i broke the component out in a custom_component. What do you think of this? Custom Component

The android app could then automatically call the service.

bdraco commented 9 months ago

@bdraco I tried something, and it works in my case. For testing in my instance i broke the component out in a custom_component. What do you think of this?

Custom Component

The android app could then automatically call the service.

It seems like if the plan is to make changes to the android app, it should instead be changed to send the name instead and avoid adding the complexity of an allow list

dshokouhi commented 9 months ago

The beacon library si not guaranteed to send the bluetooth name out. We tried that and abandoned it because it wouldn't work. More details in this comment which links to the library issue.

https://github.com/home-assistant/android/pull/2941#issuecomment-1272379540

bdraco commented 9 months ago

It probably only transmits the name if you have active scanning enabled as part of the scan response

Sorien commented 9 months ago

The beacon library si not guaranteed to send the bluetooth name out. We tried that and abandoned it because it wouldn't work. More details in this comment which links to the library issue.

home-assistant/android#2941 (comment)

it can't trasmit name couse of insufficient ble packet length, but you can send name as response to active scan, but as i see its not supported by your library, @dshokouhi you can see how to build ibeacon and send name in response here https://github.com/Sorien/blepresence/blob/master/app/src/main/java/com/sorien/ppbthome/AdvertiserService.java#L152 (its private repo but added you as colaborator)

dshokouhi commented 9 months ago

The beacon library si not guaranteed to send the bluetooth name out. We tried that and abandoned it because it wouldn't work. More details in this comment which links to the library issue. home-assistant/android#2941 (comment)

it can't trasmit name couse of insufficient ble packet length, but you can send name as response to active scan, but as i see its not supported by your library, @dshokouhi you can see how to build ibeacon and send name in response here https://github.com/Sorien/blepresence/blob/master/app/src/main/java/com/sorien/ppbthome/AdvertiserService.java#L152 (its private repo but added you as colaborator)

Thanks appreciate it. Moving to a different library is going to be quite the change actually so will take time to do so and adapt it to current features. Lots of considerations here if thats what we need to do.

In the meantime I opened a feature request for the android app to switch libraries https://github.com/home-assistant/android/issues/3948 so we can begin discussing things there in relation to the official HA android app.

chatziko commented 8 months ago

Here's a hacky but IMHO reasonable workaround proposal: apart from ibeacons with names, also allow those with some "magick" major value, say 53964. Then setup the companion app to use this major by default (and of course users can also change it themselves).

Not perfect, but it can easily solve an important use case (the companion app) with very little risk of causing trouble to users due to transient devices.

chatziko commented 8 months ago

I made a PR to try and discuss this idea. Tested it in my system, and I can finally detect my phones again :smile:

swann05 commented 7 months ago

I posted in several topics ... There are so many on that issue. As i said in my post (here https://github.com/home-assistant/android/issues/2932#issuecomment-1859135873), there are two issues. One about beacons without names not discovered and the other about HA companion not advertising a name... The second one would be really useful to use other ble "watchers" (espresense or nuki_hub for example).