Bluepy missing in new Version of Home Assistant

[Bedrock1999]

Hey!

So I am pretty new to everything here, but as far as I understand the matter, Home Assistant got an update that got rid of bluepy, which this Cluster-Light Integration relied on.

Home Assistant is recommending to switch to the Bleak library (https://pypi.org/project/bleak/). This would mean that the code has to be rewritten partially, as far as I know.

So the clusterlights.py has to be reworked I think, but I really have no idea what to do.

Would be great, if you or someone else could help with that!

[jim-rj]

I came to the same conclusions. Haven't programmed in python for ages so I am not really up to date. Seems like 7 lines of code to be updated / changed to use bleak.

@BasVerkooijen : U aren't updating the code these days getting closer and closer to december by any chance? :)

[BasVerkooijen]

Hi,

I have made an attempt to introduce Bleak. The way threading is handled in Bleak I think I now need an event loop, which is introduced in clusterlights.py.

I think something is fishy in the notification handling. I have debugged this, but see the initial expected notifications are not captured. I have added default initialization of state feedback to OFF to work around this.

Could you test this? I have tried it in Home Assistant, but I am not running the newest version yet.

See branch bleak-new-ha.

[jim-rj]

Hey @BasVerkooijen. I'll have a look at the new branch.

[jim-rj]

@BasVerkooijen, seems to work fine. Just got this in the HA Core loge file:

Logger: homeassistant.helpers.frame Source: helpers/frame.py:77 First occurred: 16:13:21 (1 occurrences) Last logged: 16:13:21

Logger: homeassistant.helpers.frame Source: helpers/frame.py:77 First occurred: 16:13:21 (1 occurrences) Last logged: 16:13:21

Detected integration that attempted to call BleakClient with an address instead of a BLEDevice. Please report issue to the custom integration author for clusterlights using this method at custom_components/clusterlights/clusterlights.py, line 70: async with BleakClient(self.mac) as self.device:

But testing for a bit i am quite impressed. Just noticed setting it to low brightness (1-3%) its still marked as being in ON but no lights are activated. However i guess this just varies from product to product.

Must say this looks very promising. I will get some more lights up and see how it works with several lights using the integration. Btw, running latest version of HA on a RPI3 with HA OS. For your information, i got another Bluetooth dongle since i saw quite many error messages from the built-in bluetooth. Since then so many more Bluetooth integrations works like a charm.

You deserve a beer for this one Bas! :)

[BasVerkooijen]

Thanks for the testing effort 👍

Good spot of the warning, I'll resolve this.

[jim-rj]

Updated info: System started to halt. Had to remove integration. Guess there is a threading / looping problem or something wrong with hci1 BT dongle :=)

Lost the connection after 30 min approx. I also have some conflicts with my bluetooth scanner integrations. No idea if it's related but included it in the logs. Do you pick which hci (bluetooth) device to use? In my logs hci1 is an extra bluetooth dongle. hci0 is the integrated RPI3 bluetooth device.

I see some bluez error messages... I think i might of left a backup folder of the old integration that might cause problems, so more testing with fresh directories + more lights.

Some logs entries:

Logger: homeassistant.components.light
Source: helpers/entity_platform.py:797
Integration: Light (documentation, issues)
First occurred: 16:58:58 (28 occurrences)
Last logged: 17:12:28

Updating clusterlights light took longer than the scheduled update interval 0:00:30

Logger: homeassistant.components.bluetooth_tracker.device_tracker
Source: components/bluetooth_tracker/device_tracker.py:72
Integration: bluetooth_tracker (documentation, issues)
First occurred: 17:05:34 (1 occurrences)
Last logged: 17:05:34

Couldn't discover bluetooth devices: error communicating with local bluetooth adapter

Logger: homeassistant.helpers.entity
Source: helpers/entity.py:527
First occurred: 16:58:38 (1 occurrences)
Last logged: 16:58:38

Update of light.veranda_cluster is taking over 10 seconds

Logger: homeassistant.components.bluetooth.scanner
Source: components/bluetooth/scanner.py:364
Integration: Bluetooth (documentation, issues)
First occurred: 16:54:58 (2 occurrences)
Last logged: 16:56:58

hci1 (XX:XX:XX:XX:XX:XX): Error stopping scanner: [org.bluez.Error.Failed] No discovery started

This error originated from a custom integration.

Logger: root
Source: custom_components/clusterlights/clusterlights.py:116
Integration: clusterlights (documentation)
First occurred: 16:53:28 (1 occurrences)
Last logged: 16:53:28

Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/config/custom_components/clusterlights/clusterlights.py", line 65, in task
    asyncio.run(self.ble_task())
  File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/config/custom_components/clusterlights/clusterlights.py", line 80, in ble_task
    await self.ble_task_loop()
  File "/config/custom_components/clusterlights/clusterlights.py", line 91, in ble_task_loop
    await self._send_packet(packet)
  File "/config/custom_components/clusterlights/clusterlights.py", line 116, in _send_packet
    await self.device.write_gatt_char(self.controlhandle, bytes(packet.data), False) # No response
  File "/usr/local/lib/python3.10/site-packages/bleak/__init__.py", line 593, in write_gatt_char
    await self._backend.write_gatt_char(char_specifier, data, response)
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/client.py", line 801, in write_gatt_char
    assert_reply(reply)
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/utils.py", line 20, in assert_reply
    raise BleakDBusError(reply.error_name, reply.body)
bleak.exc.BleakDBusError: [org.bluez.Error.Failed] Not connected

[BasVerkooijen]

Thanks for your extensive logs :slightly_smiling_face:

I see here also that the connection drops after some hours. I'll check my logs and see if I can capture the exception to reconnect when connection is dropped.

[BasVerkooijen]

I have committed a change that will reconnect to the cluster lights when a characteristic write action fails. This was also in the original code.

I'll check tomorrow if it is still connected here.

I have tested it with a test script, which seems to work. In Home Assistant, simply unplugging seems to raise some other error from Bleak for scanning for the device.

2022-11-28` 20:40:34.189 ERROR (Thread-3 (task)) [root] Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/config/custom_components/clusterlights/clusterlights.py", line 85, in task
    asyncio.run(self.ble_task())
  File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/config/custom_components/clusterlights/clusterlights.py", line 93, in ble_task
    device = await BleakScanner.find_device_by_address(self.mac, timeout=20.0)
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 221, in find_device_by_address
    return await cls.find_device_by_filter(
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 250, in find_device_by_filter
    async with cls(detection_callback=apply_filter, **kwargs):
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/scanner.py", line 96, in __aenter__
    await self.start()
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/scanner.py", line 137, in start
    self._stop = await manager.active_scan(
  File "/usr/local/lib/python3.10/site-packages/bleak/backends/bluezdbus/manager.py", line 376, in active_scan
    reply = await self._bus.call(
  File "/usr/local/lib/python3.10/site-packages/dbus_next/aio/message_bus.py", line 303, in call
    self._call(msg, reply_handler)
  File "/usr/local/lib/python3.10/site-packages/dbus_next/message_bus.py", line 588, in _call
    self.send(msg)
  File "/usr/local/lib/python3.10/site-packages/dbus_next/aio/message_bus.py", line 326, in send
    self._writer.schedule_write(msg, future)
  File "/usr/local/lib/python3.10/site-packages/dbus_next/aio/message_bus.py", line 85, in schedule_write
    self.loop.add_writer(self.fd, self.write_callback)
  File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 341, in add_writer
    self._add_writer(fd, callback, *args)
  File "/usr/local/lib/python3.10/asyncio/selector_events.py", line 290, in _add_writer
    self._check_closed()
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 515, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

[jim-rj]

I'll setup a separate testing setup for this in a couple of days and have a look :)

[BasVerkooijen]

I have pushed a new commit that resolves the reconnect issue at my end. This commit marks the ble_task invoking bleak as an asyncio event loop. This should make sure bleak events can run to completion before ble_task is closed.

I have this running now for a few days and the cluster light controls are still working.

My next step would be improving the state reporting.

[jim-rj]

Hey. Finally got a testing setup up and running.

Don't know if I am doing something wrong or if its related to running HA 2022.12 (was HA 2022.11.4 last time i got it working with the first version of the branch) and/or new updates to the branch.

Have made sure i cloned the new branch, so old files shouldn't linger around. Added the lights in configuration.yaml and rebooted several times. Tried using separate include file as well... pasting config here... just in case... Documentation said to use "AA:BB:CC..." for the mac. But I have tried with both " and '. Could you give me an example from your configuration file and i'll try again. How many lights have you tried to run this with?

Will try and get my linux box up and running again to get better into the code, its a pain to sit on a windows machine :) Hope i get the time to contribute.

Great work still working on this :)

light:
  - platform: group
    name: RGB Lights
    entities:
      - [just some lights]
      - [just some lights]

  - platform: clusterlights
    devices:
     '24:35:CC:0C:XX:XX':
       name: Cluster Veranda
     '24:35:CC:0C:XX:XX':
       name: Cluster Office
     '24:35:CC:0C:XX:XX':
       name: Cluster Living Room

Got some weird logs, assume they are all related to the last one:

Logger: homeassistant.components.light
Source: helpers/entity_platform.py:797
Integration: Light ([documentation](https://www.home-assistant.io/integrations/light), [issues](https://github.com/home-assistant/home-assistant/issues?q=is%3Aissue+is%3Aopen+label%3A%22integration%3A+light%22))
First occurred: 11:32:17 (45 occurrences)
Last logged: 11:54:26

Updating clusterlights light took longer than the scheduled update interval 0:00:30

Logger: homeassistant.helpers.entity
Source: helpers/entity.py:527
First occurred: 11:31:57 (2 occurrences)
Last logged: 11:32:56

Update of light.veranda_cluster is taking over 10 seconds

Logger: homeassistant.core
Source: core.py:331
First occurred: 11:32:25 (1 occurrences)
Last logged: 11:32:25

Something is blocking Home Assistant from wrapping up the start up phase. We're going to continue anyway. Please report the following info at https://github.com/home-assistant/core/issues: [..] (Long list of all the integrations, with clusterlights in it, pretty sure i didnt have this before,

Logger: root
Source: custom_components/clusterlights/clusterlights.py:99
Integration: clusterlights ([documentation](https://github.com/BasVerkooijen/cluster-lights-home-assistant))
First occurred: 11:26:20 (3 occurrences)
Last logged: 11:26:20

Uncaught thread exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/config/custom_components/clusterlights/clusterlights.py", line 89, in task
    loop.run_until_complete(self.ble_task())
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/config/custom_components/clusterlights/clusterlights.py", line 99, in ble_task
    device = await BleakScanner.find_device_by_address(self.mac, timeout=20.0)
AttributeError: type object 'HaBleakScannerWrapper' has no attribute 'find_device_by_address'

[Bedrock1999]

For me, it seems to add the entity, and sometimes I can even interact. However, I also have the problem, that the light does not do anything when I interact with the entity and looking into the logs it also gives these “took more than 30s” messages:

2023-01-13 12:40:26.314 WARNING (MainThread) [homeassistant.components.light] Updating clusterlights light took longer than the scheduled update interval 0:00:30

2023-01-13 12:40:56.314 WARNING (MainThread) [homeassistant.components.light] Updating clusterlights light took longer than the scheduled update interval 0:00:30

I am however not 100% sure if the Bluetooth antenna of my cluster lights is just so bad, that the antenna connected to my Raspberry Pi isn't always reaching it.

[NickdeK]

~I just want to report that the bleak-new-ha branch seems to be working fine for me with the latest HA version. Great work, thanks for this!~

edit: seems I run into the same issue with the "took more then 30s" logs after some time.