sopelj / hass-ember-mug-component

Ember Mug Integration for Home Assistant
MIT License
116 stars 4 forks source link

Bluetooth proxies are not working #15

Closed bakerboy908 closed 1 year ago

bakerboy908 commented 2 years ago

Im unsure if this is an issue with the ESP proxy or with this integration

Here is the error im getting in home assistant (C3:DF:EA:CC:38:40) - C3:DF:EA:CC:38:40 -> esp32-bluetooth-proxy-c99384: Failed to connect: No backend with an available connection slot that can reach address C3:DF:EA:CC:38:40 was found: The proxy/adapter is out of connection slots; Add additional proxies near this device Pairing is not available in ESPHome.

bakerboy908 commented 2 years ago

This is the esp home bluetooth proxy logs when it attempts to connect


[I][esp32_ble_client:058]: Attempting BLE connection to c3:df:ea:cc:38:40
[I][esp32_ble_client:142]: Service UUID: 0x1800
[I][esp32_ble_client:143]:   start_handle: 0x1  end_handle: 0x7
[I][esp32_ble_client.service:057]:  characteristic 0x2A00, handle 0x3, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic 0x2A01, handle 0x5, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic 0x2A04, handle 0x7, properties 0x2
[I][esp32_ble_client:142]: Service UUID: 0x1801
[I][esp32_ble_client:143]:   start_handle: 0x8  end_handle: 0xb
[I][esp32_ble_client.service:057]:  characteristic 0x2A05, handle 0xa, properties 0x20
[I][esp32_ble_client:142]: Service UUID: FC543622-236C-4C94-8FA9-944A3E5353FA
[I][esp32_ble_client:143]:   start_handle: 0xc  end_handle: 0x33
[I][esp32_ble_client.service:057]:  characteristic FC540001-236C-4C94-8FA9-944A3E5353FA, handle 0xe, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540002-236C-4C94-8FA9-944A3E5353FA, handle 0x10, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC540003-236C-4C94-8FA9-944A3E5353FA, handle 0x12, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540004-236C-4C94-8FA9-944A3E5353FA, handle 0x14, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540005-236C-4C94-8FA9-944A3E5353FA, handle 0x16, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC540006-236C-4C94-8FA9-944A3E5353FA, handle 0x18, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540007-236C-4C94-8FA9-944A3E5353FA, handle 0x1a, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC540008-236C-4C94-8FA9-944A3E5353FA, handle 0x1c, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC54000A-236C-4C94-8FA9-944A3E5353FA, handle 0x1e, properties 0x8
[I][esp32_ble_client.service:057]:  characteristic FC54000C-236C-4C94-8FA9-944A3E5353FA, handle 0x20, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC54000D-236C-4C94-8FA9-944A3E5353FA, handle 0x22, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC54000E-236C-4C94-8FA9-944A3E5353FA, handle 0x24, properties 0x2
[I][esp32_ble_client.service:057]:  characteristic FC54000F-236C-4C94-8FA9-944A3E5353FA, handle 0x26, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540010-236C-4C94-8FA9-944A3E5353FA, handle 0x28, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540011-236C-4C94-8FA9-944A3E5353FA, handle 0x2a, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540012-236C-4C94-8FA9-944A3E5353FA, handle 0x2d, properties 0x10
[I][esp32_ble_client.service:057]:  characteristic FC540014-236C-4C94-8FA9-944A3E5353FA, handle 0x30, properties 0xa
[I][esp32_ble_client.service:057]:  characteristic FC540013-236C-4C94-8FA9-944A3E5353FA, handle 0x32, properties 0x10
[I][esp32_ble_client:142]: Service UUID: 00001530-1212-EFDE-1523-785FEABCD123
[I][esp32_ble_client:143]:   start_handle: 0x34  end_handle: 0xffff
[I][esp32_ble_client.service:057]:  characteristic 00001532-1212-EFDE-1523-785FEABCD123, handle 0x36, properties 0x4
[I][esp32_ble_client.service:057]:  characteristic 00001531-1212-EFDE-1523-785FEABCD123, handle 0x38, properties 0x18
[I][esp32_ble_client.service:057]:  characteristic 00001534-1212-EFDE-1523-785FEABCD123, handle 0x3b, properties 0x2```
sopelj commented 2 years ago

Hi, I don't have any proxies set up yet, but there do seem to be similar issues with other integrations. a comment on the Yale Lock integration seems to indicate it's fixed with the latest home assistant and proxy firmware. Are yours up to date? Do you have any other Bluetooth devices setup?

bakerboy908 commented 2 years ago

They havent released 2022.11.0 yet, but i tried the dev branch, with no luck

bakerboy908 commented 2 years ago

Ok i think i hadnt setup the dev branch correctly, the error has changed to parring not available in ESPHome


Source: custom_components/ember_mug/coordinator.py:88
Integration: Ember Mug ([documentation](https://github.com/sopelj/hass_ember_mug), [issues](https://github.com/sopelj/hass_ember_mug/issues))
First occurred: 10:30:29 (5 occurrences)
Last logged: 10:37:04

Pairing is not available in ESPHome.```
janpgu commented 2 years ago

Hi all, First of all: thanks for the great work @sopelj :) Do we have any updates on the issue mentioned by @bakerboy908 ? I'm having the same issue (also using EspHome active bluetooth proxy).

sopelj commented 2 years ago

Hi and Thanks! Sorry for the late response.

The problem is that, as the error states, pairing doesn't seem to be supported with Bluetooth Proxies currently. And the mug needs to be paired to get kicked out of it's pairing mode. Otherwise, it will stay there until the user hits the button.

Until the proxies support pairing, I'm trying to find a way to skip the pairing if we're in the context of a proxy, but I haven't figured out how to detect it yet. The worst part is, if it simply raised a Bluetooth error it would be fine, but it seems to fail at another level that I can't catch.

sopelj commented 2 years ago

Ok, I think I found a way to catch this error. However, if it works you might need to manually force it out of pairing mode by pressing the button once, once its setup. I don't have proxies to test it right now, but I pushed a beta version 0.4.1 if someone has a chance to test it would be much appreciated. Thanks!

belenkiy-lab commented 2 years ago

I have a proxy and a mug. download version 0.4.1-beta-1 add mug from integration page then canceled the pairing mode on the mugs after that In the esp proxy console:

16:18:05.819 -> [esp32_ble_client:036]: Found device at MAC address [DB:7F:78:E1:8A:50]
16:18:05.819 -> [esp32_ble_client:058]: Attempting BLE connection to db:7f:78:e1:8a:50
16:18:06.051 -> lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
16:18:06.051 -> [esp32_ble_client:089]: connect to 00:00:00:00:00:00 failed, status=133
16:18:06.051 -> [esp32_ble_tracker:264]: Starting scan...
16:18:07.113 -> [esp32_ble_client:036]: Found device at MAC address [DB:7F:78:E1:8A:50]
16:18:07.113 -> [esp32_ble_client:058]: Attempting BLE connection to db:7f:78:e1:8a:50
16:18:07.252 -> lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
16:18:07.298 -> [esp32_ble_client:089]: connect to 00:00:00:00:00:00 failed, status=133
16:18:07.298 -> [esp32_ble_tracker:264]: Starting scan...
16:18:08.275 -> [esp32_ble_client:036]: Found device at MAC address [DB:7F:78:E1:8A:50]
16:18:08.322 -> [esp32_ble_client:058]: Attempting BLE connection to db:7f:78:e1:8a:50
16:18:08.415 -> lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
16:18:08.461 -> [esp32_ble_client:089]: connect to 00:00:00:00:00:00 failed, status=133
16:18:08.461 -> [esp32_ble_tracker:264]: Starting scan...
16:18:09.521 -> [esp32_ble_client:036]: Found device at MAC address [DB:7F:78:E1:8A:50]
16:18:09.566 -> [esp32_ble_client:058]: Attempting BLE connection to db:7f:78:e1:8a:50
16:18:09.708 -> lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
16:18:09.755 -> [esp32_ble_client:089]: connect to 00:00:00:00:00:00 failed, status=133
16:18:09.755 -> [esp32_ble_tracker:264]: Starting scan...
16:18:10.917 -> [esp32_ble_client:036]: Found device at MAC address [DB:7F:78:E1:8A:50]
16:18:10.917 -> [esp32_ble_client:058]: Attempting BLE connection to db:7f:78:e1:8a:50
16:18:11.151 -> lld_pdu_get_tx_flush_nb HCI packet count mismatch (0, 1)
16:18:11.198 -> [esp32_ble_client:089]: connect to 00:00:00:00:00:00 failed, status=133

in the ha log

This error originated from a custom integration.
Logger: custom_components.ember_mug.coordinator
Source: custom_components/ember_mug/coordinator.py:88
Integration: Ember Mug ([documentation](https://github.com/sopelj/hass_ember_mug), [issues](https://github.com/sopelj/hass_ember_mug/issues))
First occurred: 16:17:55 (7 occurrences)
Last logged: 16:25:19

(DB:7F:78:E1:8A:50) - DB:7F:78:E1:8A:50 -> esp32-bluetooth-proxy-4d48d4: Failed to connect: Error ESP_GATT_CONN_FAIL_ESTABLISH while connecting: Connection failed to establish: Interference/range; External Bluetooth adapter w/extension may help; Extension cables reduce USB 3 port interference
(DB:7F:78:E1:8A:50) - DB:7F:78:E1:8A:50 -> esp32-bluetooth-proxy-4d48d4: Failed to connect: No backend with an available connection slot that can reach address DB:7F:78:E1:8A:50 was found: The proxy/adapter is out of connection slots; Add additional proxies near this device
cadwizzard commented 1 year ago

I have the same problem with BT proxies. The mug worked fine when near to the adapter in the Pi, but was out of range in the kitchen. I have an ESP32-POE-ISO-EA which seems to be working fine, updated through ESPhome to latest firmware. I have updated to the beta Ember integration and not noticed much difference.

Some log info, from ESPhome: [13:54:14][D][esp32_ble_tracker:264]: Starting scan... [13:54:15][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:15][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:15][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [13:54:15][D][esp32_ble_tracker:264]: Starting scan... [13:54:16][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:16][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:16][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [13:54:16][D][esp32_ble_tracker:264]: Starting scan... [13:54:17][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:17][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:18][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [13:54:18][D][esp32_ble_tracker:264]: Starting scan... [13:54:19][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:19][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:19][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [13:54:19][D][esp32_ble_tracker:264]: Starting scan... [13:54:20][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:20][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:20][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [13:54:20][D][esp32_ble_tracker:264]: Starting scan...

Log from HA: `Logger: custom_components.ember_mug.coordinator Source: custom_components/ember_mug/coordinator.py:88 Integration: Ember Mug (documentation, issues) First occurred: 13:48:55 (7 occurrences) Last logged: 13:54:21

(FA:71:15:24:9F:85) - FA:71:15:24:9F:85 -> /org/bluez/hci1: Failed to connect: (FA:71:15:24:9F:85) - FA:71:15:24:9F:85 -> esp32-bluetooth-proxy-0464bc: Failed to connect: Error ESP_GATT_CONN_FAIL_ESTABLISH while connecting: Connection failed to establish: Interference/range; External Bluetooth adapter w/extension may help; Extension cables reduce USB 3 port interference`

What I see in the HA UI for the integration when the mug ISNT in pairing mode: Retrying setup: An error occurred updating mug: e=BleakAbortedError(' (FA:71:15:24:9F:85) - FA:71:15:24:9F:85 -> esp32-bluetooth-proxy-0464bc: Failed to connect: Error

I see this in the logs when the mug IS in pairing mode: "Bluetooth GATT Error address=FA:71:15:24:9F:85 handle=36 error=5 description=Insufficient authentication"

In the proxy/ESP32 logs when the mug IS in pairing mode, I see this: `INFO Reading configuration /config/esphome/esp32-bluetooth-proxy-0464bc.yaml... INFO Starting log output from esp32-bluetooth-proxy-0464bc.local using esphome API INFO Successfully connected to esp32-bluetooth-proxy-0464bc.local [14:15:19][I][app:102]: ESPHome version 2022.11.4 compiled on Dec 5 2022, 13:17:22 [14:15:19][I][app:104]: Project esphome.bluetooth-proxy version 1.0

[14:15:20][C][logger:294]: Level: DEBUG [14:15:20][C][logger:295]: Log Baud Rate: 115200 [14:15:20][C][logger:296]: Hardware UART: UART0 [14:15:20][C][bluetooth_proxy:051]: Bluetooth Proxy: [14:15:20][C][bluetooth_proxy:052]: Active: YES [14:15:20][C][safe_mode.button:022]: Safe Mode Button 'Safe Mode Boot'

[14:15:20][C][esp32_ble_tracker:796]: BLE Tracker: [14:15:20][C][esp32_ble_tracker:797]: Scan Duration: 300 s [14:15:20][C][esp32_ble_tracker:798]: Scan Interval: 1100.0 ms [14:15:20][C][esp32_ble_tracker:799]: Scan Window: 1100.0 ms [14:15:20][C][esp32_ble_tracker:800]: Scan Type: ACTIVE [14:15:20][C][esp32_ble_tracker:801]: Continuous Scanning: True [14:15:20][C][captive_portal:088]: Captive Portal:

[14:15:20][C][mdns:104]: Hostname: esp32-bluetooth-proxy-0464bc [14:15:20][C][ota:093]: Over-The-Air Updates: [14:15:20][C][ota:094]: Address: esp32-bluetooth-proxy-0464bc.local:3232FA9-944A [14:15:20][C][api:138]: API Server: [14:15:20][C][api:139]: Address: esp32-bluetooth-proxy-0464bc.local:6053 [14:15:20][C][api:143]: Using noise encryption: NO [14:15:20][C][improv_serial:032]: Improv Serial: [14:16:06][I][esp32_ble_client:066]: [0] [fa:71:15:24:9f:85] Disconnecting. [14:16:06][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [14:16:06][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [14:16:07][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [14:16:07][D][esp32_ble_tracker:264]: Starting scan... [14:16:07][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [14:16:07][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [14:16:09][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 0x1800 [14:16:09][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x1 end_handle: 0x7 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A00, handle 0x3, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A01, handle 0x5, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A04, handle 0x7, properties 0x2 [14:16:09][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 0x1801 [14:16:09][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x8 end_handle: 0xb [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A05, handle 0xa, properties 0x20 [14:16:09][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: FC543622-236C-4C94-8FA9-944A3E5353FA [14:16:09][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0xc end_handle: 0x33 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540001-236C-4C94-8FA9-944A3E5353FA, handle 0xe, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540002-236C-4C94-8FA9-944A3E5353FA, handle 0x10, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540003-236C-4C94-8FA9-944A3E5353FA, handle 0x12, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540004-236C-4C94-8FA9-944A3E5353FA, handle 0x14, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540005-236C-4C94-83E5353FA, handle 0x16, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540006-236C-4C94-8FA9-944A3E5353FA, handle 0x18, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540007-236C-4C94-8FA9-944A3E5353FA, handle 0x1a, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540008-236C-4C94-8FA9-944A3E5353FA, handle 0x1c, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000A-236C-4C94-8FA9-944A3E5353FA, handle 0x1e, properties 0x8 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000C-236C-4C94-8FA9-944A3E5353FA, handle 0x20, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000D-236C-4C94-8FA9-944A3E5353FA, handle 0x22, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000E-236C-4C94-8FA9-944A3E5353FA, handle 0x24, properties 0x2 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000F-236C-4C94-8FA9-944A3E5353FA, handle 0x26, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540010-236C-4C94-8FA9-944A3E5353FA, handle 0x28, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540011-236C-4C94-8FA9-944A3E5353FA, handle 0x2a, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540012-236C-4C94-8FA9-944A3E5353FA, handle 0x2d, properties 0x10 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540014-236C-4C94-8FA9-944A3E5353FA, handle 0x30, properties 0xa [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540013-236C-4C94-8FA9-944A3E5353FA, handle 0x32, properties 0x10 [14:16:09][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 00001530-1212-EFDE-1523-785FEABCD123 [14:16:09][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x34 end_handle: 0xffff [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001532-1212-EFDE-1523-785FEABCD123, handle 0x36, properties 0x4 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001531-1212-EFDE-1523-785FEABCD123, handle 0x38, properties 0x18 [14:16:09][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001534-1212-EFDE-1523-785FEABCD123, handle 0x3b, properties 0x2 [14:16:09][D][esp32_ble_tracker:264]: Starting scan... [14:16:58][I][esp32_ble_client:066]: [0] [fa:71:15:24:9f:85] Disconnecting. [14:16:58][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [14:16:58][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [14:16:58][W][esp32_ble_client:106]: [0] [] Connection failed, status=133 [14:16:58][D][esp32_ble_tracker:264]: Starting scan... [14:16:59][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [14:16:59][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [14:17:01][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 0x1800 [14:17:01][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x1 end_handle: 0x7 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A00, handle 0x3, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A01, handle 0x5, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A04, handle 0x7, properties 0x2 [14:17:01][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 0x1801 [14:17:01][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x8 end_handle: 0xb [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 0x2A05, handle 0xa, properties 0x20 [14:17:01][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: FC543622-236C-4C94-8FA9-944A3E5353FA [14:17:01][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0xc end_handle: 0x33 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540001-236C-4C94-8FA9-944A3E5353FA, handle 0xe, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540002-236C-4C94-8FA9-944A3E5353FA, handle 0x10, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540003-236C-4C94-8FA9-944A3E5353FA, handle 0x12, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540004-236C-4C94-8FA9-944A3E5353FA, handle 0x14, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540005-236C-4C94-8FA9-944A3E5353FA, handle 0x16, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540006-236C-4C94-8FA9-944A3E5353FA, handle 0x18, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540007-236C-4C94-8FA9-944A3E5353FA, handle 0x1a, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540008-236C-4C94-8FA9-944A3E5353FA, handle 0x1c, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000A-236C-4C94-8FA9-944A3E5353FA, handle 0x1e, properties 0x8 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000C-236C-4C94-8FA9-944A3E5353FA, handle 0x20, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000D-236C-4C94-8FA9-944A3E5353FA, handle 0x22, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000E-236C-4C94-8FA9-944A3E5353FA, handle 0x24, properties 0x2 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC54000F-236C-4C94-8FA9-944A3E5353FA, handle 0x26, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540010-236C-4C94-8FA9-944A3E5353FA, handle 0x28, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540011-236C-4C94-8FA9-944A3E5353FA, handle 0x2a, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540012-236C-4C94-8FA9-944A3E5353FA, handle 0x2d, properties 0x10 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540014-236C-4C94-8FA9-944A3E5353FA, handle 0x30, properties 0xa [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic FC540013-236C-4C94-8FA9-944A3E5353FA, handle 0x32, properties 0x10 [14:17:01][I][esp32_ble_client:154]: [0] [fa:71:15:24:9f:85] Service UUID: 00001530-1212-EFDE-1523-785FEABCD123 [14:17:01][I][esp32_ble_client:156]: [0] [fa:71:15:24:9f:85] start_handle: 0x34 end_handle: 0xffff [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001532-1212-EFDE-1523-785FEABCD123, handle 0x36, properties 0x4 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001531-1212-EFDE-1523-785FEABCD123, handle 0x38, properties 0x18 [14:17:01][I][esp32_ble_client:059]: [0] [fa:71:15:24:9f:85] characteristic 00001534-1212-EFDE-1523-785FEABCD123, handle 0x3b, properties 0x2 [14:17:01][D][esp32_ble_tracker:264]: Starting scan... [14:17:02][W][bluetooth_proxy.connection:054]: [0] [fa:71:15:24:9f:85] Error reading char/descriptor at handle 0x24, status=5 `

The BT proxy is in the same room and I have tried 0.5m away and 4m away from the mug with same results. and the proxy is on a table away from any interference.

I did try untrusting the mug from the adapter settings via VNC in the Pi bluez GUI in case is was conflicting, but that didnt help.

Is there a way to 'pair' the mug on another adapter and it still work when then brought into the range of the proxy?. Like I mentioned it was working previously close to the other adapte,r but was out of range where it is normally located. I added the proxy and located it nearby but it never connected (no available slot errors, thats why I untrusted it from the other adapter in case it was trying to make 2 connections etc).

I did try taking the mug back to the BT adapter connected to the Pi/HA and it was redetected perfectly straight away. But then moving it back near the proxy adapter i just see the connection errors in the logs like: [13:54:20][D][esp32_ble_client:039]: [0] [fa:71:15:24:9f:85] Found device [13:54:20][I][esp32_ble_client:054]: [0] [fa:71:15:24:9f:85] Attempting BLE connection [13:54:20][W][esp32_ble_client:106]: [0] [] Connection failed, status=133]

And of course then the entities for the mug just stop updating,

I'm happy to try anything to get it working :)

cadwizzard commented 1 year ago

I'll thought I'd add, I noticed the update for esphome today while I was out and installed it. When I got in, the mug was working over the proxy. Right up until I made drink & took it into another room out of range, then it was back to not connecting again when I brought it back into range. I saw an update in esphome for my BT proxy firmware. Installed that but just see the ble generic connection errors again. Unsure what triggered the mug to connect earlier by itself seemingly while I was out, but looks like there is hope. I haven't put the mug into pairing mode at all today

sopelj commented 1 year ago

Thank you very much for all the tests and detailed logs. Very much appreciated! Sorry, I've been quite busy the past couple weeks. That is quite encouraging that it was working over the proxy. My mug occasionally disconnects after a certain period of inactivity even without a proxy, so it may not be related to the proxy even. I've been investigating alternative methods to make the connection more reliable. The new version of home assistant has upgraded the libraries for Bluetooth and so far seems a bit more reliable. I'm trying to read through the code to see if anything new has been added to support this specifically. I'll try and set up a proxy this weekend if I can find an unused ESP32.

winstona commented 1 year ago

After a bit of poking around it seems if you hack out querying dsk, it seems to at least "pair" and pull data through - ie something like:

    async def get_dsk(self) -> str:
        """Get mug dsk from gatt."""
        return "c29tZXRoaW5nIGVsc2U="
        # return decode_byte_string(await self._client.read_gatt_char(UUID_DSK))

I've seen some weird combination of issues getting to this point though. My current setup:

I noticed that esphome 2022.11.5 would trip up attempting to get the UUID_MUG_ID characteristic and fail out. Bumping esphome to 2022.12.0 would not fail in the same place, however when attempting to add the mug, it would consistently crash the ESP32 and reboot it completely (possibly a OOM issue?).

Trying to observe UUID_DSK using a utility like Bluetility looks like it may require an explicit pair (which it looks like pairing is not available in HA/esphome bluetooth proxy yet at all) as it pops up with a pairing request for me, whereas some other characteristics can be pulled without pairing.

I'll still be testing this out a bit more, but one issue I already noticed is that it appears as soon as the esp32 or HA restarts I have to put it back in pairing mode to re-initiate the connection (but it does not need to be explicitly re-added in HA)

I'm not really familiar with the bluetooth stack in depth, but any ideas on a way to fake the pairing to get the mug to trust the bluetooth proxy?

cadwizzard commented 1 year ago

I've updated both HA and ESP to the latest versions. I do see different connection strings but the connection still fails. I caught this just after the ESP update Screenshot_20221214_081943_Home Assistant If I take the mug without trying to pair or anything else near to the non proxy BT adapter it connects fine, but still doesnt when in range of only the proxy. The error in HA remains the same as before the HA/ESP updates: (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Error ESP_GATT_CONN_FAIL_ESTABLISH while connecting: Connection failed to establish: Interference/range; External Bluetooth adapter w/extension may help; Extension cables reduce USB 3 port interference (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Timeout waiting for connect response while connecting to FA:71:15:24:9F:85 after 20.0s, disconnect timed out: False, after 5.0s

One thing in the ESP/BT Proxy notes relates to performance being better if physically flashed rather than OTA - that I havent done yet, as the adapter is PoE connected in the loft and its -5 degrees up there, so waiting a good reason to go balance on the joists :) If there is a likely chance that will make a difference, i'll go fetch the ESP32 and do that too.

sopelj commented 1 year ago

Oh wow! Much appreciated. @winstona I have not had the chance to test the Ember Mug v1. Does it work without bluetooth proxies? If not, I wonder if the UUIDs are slightly different. Yeah, I have not been able to understand the magic that UDSK and DSK are used for, but from what I can tell it seems to be some sort of auth. They are set by the Ember app and seem to be generated on their server post login. If they are not correctly setting values on the mug fail. Has your mug been setup in their app before attempting to use it with this integration? That seems to set the values correctly.

sopelj commented 1 year ago

@cadwizzard Hmm, that's too bad. Thanks for testing. I get not wanting to go up there flash it. PoE powered is a good idea. I'm not sure what the range is on Bluetooth Proxies either. Does it work well with any other devices? Thanks! I'll try too.

winstona commented 1 year ago

I haven't had a chance to test without a bluetooth proxy yet, as passing bt through to the home assistant docker container was a bit messier than I hoped for - but I will try to test that out.

Like your speculation, my suspicion is that the UUIDs are the same, as I get all other data correctly (current temp, target temp, battery %, set temp, fill level, heating/cooling/perfect status, etc). I originally was thinking it could be a firmware issue as I was running an older version (I had issues getting it upgraded) but I finally got it upgraded to the latest firmware version and it didn't seem to make a difference.

I have 2 v1's, both were connected in the app before connecting to the integration, both stayed connected over a ~24hr period so far except for when the battery died on the mug or the ESP was restarted.

Does this sound like behavior that could be seen if the Ember app didn't set the DSK correctly? I am getting a successful value for UDSK without issue it appears.

@cadwizzard it appears you're using esp-idf framework on that ESP - I forgot to specify but I'm using arduino framework. I'll try to swap mine over to esp-idf and test things out again since my ESP is far more accessible.

cadwizzard commented 1 year ago

@winstona I'm using the 'easy' install from here https://esphome.github.io/bluetooth-proxies/ and choosing the olimex option, then just updating in HA/ESPhome add-in from there. I flashed by serial the first time, and OTA update from that point on. As of today the version of ESPhome 2022.12.1 was available and i've updated the BT ESP32. The connection tried from the device and errors are now: `[16:13:30][C][esp32_ble_tracker:875]: Scan Duration: 300 s [16:13:30][C][esp32_ble_tracker:876]: Scan Interval: 1100.0 ms [16:13:30][C][esp32_ble_tracker:877]: Scan Window: 1100.0 ms [16:13:30][C][esp32_ble_tracker:878]: Scan Type: ACTIVE [16:13:30][C][esp32_ble_tracker:879]: Continuous Scanning: True

[16:13:30][C][mdns:104]: Hostname: esp32-bluetooth-proxy-0464bc [16:13:30][C][ota:093]: Over-The-Air Updates: [16:13:30][C][ota:094]: Address: esp32-bluetooth-proxy-0464bc.local:3232 [16:13:30][C][api:138]: API Server: [16:13:30][C][api:139]: Address: esp32-bluetooth-proxy-0464bc.local:6053 [16:13:30][C][api:143]: Using noise encryption: NO [16:13:30][C][improv_serial:032]: Improv Serial: [16:13:33][D][api:102]: Accepted ::FFFF:192.168.1.249 [16:13:33][D][api.connection:918]: Home Assistant 2022.12.6 (::FFFF:192.168.1.249): Connected successfully [16:13:40][I][bluetooth_proxy:254]: [0] [FA:71:15:24:9F:85] Connecting v3 without cache [16:13:40][D][esp32_ble_tracker:216]: Pausing scan to make connection... [16:13:40][I][esp32_ble_client:064]: [0] [FA:71:15:24:9F:85] 0x01 Attempting BLE connection [16:13:45][D][esp-idf:000]: W (27115) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[16:13:49][D][esp-idf:000]: W (31120) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[16:13:53][D][esp-idf:000]: W (35125) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[16:13:55][D][esp-idf:000]: W (37136) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[16:13:55][D][esp-idf:000]: W (37142) BT_APPL: gattc_conn_cb: if=3 st=0 id=3 rsn=0x3e

[16:13:55][D][esp-idf:000]: W (37145) BT_APPL: gattc_conn_cb: if=4 st=0 id=4 rsn=0x3e

[16:13:55][D][esp-idf:000]: W (37147) BT_APPL: gattc_conn_cb: if=5 st=0 id=5 rsn=0x3e

[16:13:55][W][esp32_ble_client:131]: [0] [] Connection failed, status=133 [16:13:55][D][esp32_ble_tracker:327]: Starting scan... [16:13:56][I][bluetooth_proxy:254]: [0] [FA:71:15:24:9F:85] Connecting v3 without cache [16:13:56][D][esp32_ble_tracker:216]: Pausing scan to make connection... [16:13:56][I][esp32_ble_client:064]: [0] [FA:71:15:24:9F:85] 0x01 Attempting BLE connection [16:13:57][D][esp-idf:000]: W (39145) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e ` @sopelj I see other devices detected (pretty sure my Hue bulbs mainly, but I have added them as I use the zigbee hub with them outside of HA). The only reason I got the BT proxy was for HA to see my Ember mug in the kitchen :)

winstona commented 1 year ago

ok after switching to esp-idf I'm getting very similar output to you @cadwizzard (everything else appears to be the same as in the arduino framework) - can you try holding down the button on the mug to get it into pairing mode (you shouldn't need to do anything else) and see if it connects and starts sending data? Once you see data in HA, you should be able to press the button once to kick it out of pairing mode and it should stay connected. I'm curious to see if we're seeing more or less the same results.

cadwizzard commented 1 year ago

If I put it in pairing mode I still see errors in the logs, but they relate to the pairing/authentication: (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Error ESP_GATT_CONN_FAIL_ESTABLISH while connecting: Connection failed to establish: Interference/range; External Bluetooth adapter w/extension may help; Extension cables reduce USB 3 port interference (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Timeout waiting for connect response while connecting to FA:71:15:24:9F:85 after 20.0s, disconnect timed out: False, after 5.0s (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Bluetooth GATT Error address=FA:71:15:24:9F:85 handle=36 error=5 description=Insufficient authentication ember1

winstona commented 1 year ago

ah sorry a few things I forgot to specify - I remember seeing that error, and I think ultimately it's due to the DSK uuid needing a pair to read. I hacked something together, which you can test out by using my branch temporarily - if you go into your custom_components/ember_mug/manifest.json and modify the requirements to include the repo/branch, kind of like this:

{
  "codeowners": ["@sopelj"],
  "config_flow": true,
  "documentation": "https://github.com/sopelj/hass_ember_mug",
  "issue_tracker": "https://github.com/sopelj/hass_ember_mug/issues",
  "domain": "ember_mug",
  "name": "Ember Mug",
  "version": "0.4.1",
  "iot_class": "local_polling",
  "dependencies": ["bluetooth"],
  "requirements": ["git+https://github.com/winstona/python-ember-mug.git@override_dsk#python-ember-mug==0.4.2"],
  "bluetooth": [
    { "local_name": "Ember Ceramic Mug" },
    { "service_uuid": "fc543622-236c-4c94-8fa9-944a3e5353fa" }
  ]
}

and restart HA - you may have to kill the docker container (if you're running docker) and restart it to pull the new branch. Of course if you have any concerns you can look at the commit history on that branch to see what has changed.

You also want to keep an eye on the HA logs, as it will output errors/details there sometimes. I turned up the logging for that component as well to get a bit more info by adding this to my configuration.yaml in HA:

logger:
  default: warning
  logs:
    custom_components.ember_mug: debug

I think I also hacked some log messages into my HA component temporarily, but I don't think there was anything useful there that I remember.

Finally, I noticed just a bit ago that if it loses connection, holding the button until pair mode, I only had to wait < 1s, press the button to get it out of pair mode again and it was connected and sending data. It'd be annoying to have to hold to pair every time it loses connection (at least until bluetooth_proxy adds pairing support), but it could be better than nothing at all.

sopelj commented 1 year ago

Oh wow, thank you for your tests, they are much appreciated. Ok, I just don't understand why the DSK would be the only attribute to cause problems, though. It isn't actually used currently by the integration since I haven't figured how to generate the value for the udsk anyway, so I could either remove them or catch read errors for it and just log a warning. I'll try setting up a proxy tomorrow and testing this too.

cadwizzard commented 1 year ago

Since the latest .2 increment of ESPhome, I see this now in the logs of the proxy adapter: Doesnt seem to matter if the mugs in pairing mode or not: `[09:47:14][W][esp32_ble_client:131]: [0] [] Connection failed, status=133 [09:47:14][D][esp32_ble_tracker:327]: Starting scan... [09:47:14][I][bluetooth_proxy:250]: [0] [FA:71:15:24:9F:85] Connecting v3 with cache [09:47:15][D][esp32_ble_tracker:216]: Pausing scan to make connection... [09:47:15][I][esp32_ble_client:064]: [0] [FA:71:15:24:9F:85] 0x01 Attempting BLE connection [09:47:15][D][esp-idf:000]: W (101278) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[09:47:15][D][esp-idf:000]: W (101446) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[09:47:15][D][esp-idf:000]: W (101561) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[09:47:15][D][esp-idf:000]: W (101664) BT_HCI: hcif disc complete: hdl 0x0, rsn 0x3e

[09:47:15][D][esp-idf:000]: W (101668) BT_APPL: gattc_conn_cb: if=3 st=0 id=3 rsn=0x3e

[09:47:15][D][esp-idf:000]: W (101670) BT_APPL: gattc_conn_cb: if=4 st=0 id=4 rsn=0x3e

[09:47:15][D][esp-idf:000]: W (101671) BT_APPL: gattc_conn_cb: if=5 st=0 id=5 rsn=0x3e

[09:47:15][W][esp32_ble_client:131]: [0] [] Connection failed, status=133 [09:47:15][D][esp32_ble_tracker:327]: Starting scan... [09:48:36][I][bluetooth_proxy:250]: [0] [FA:71:15:24:9F:85] Connecting v3 with cache [09:48:36][D][esp32_ble_tracker:216]: Pausing scan to make connection... [09:48:36][I][esp32_ble_client:064]: [0] [FA:71:15:24:9F:85] 0x01 Attempting BLE connection [09:48:36][D][esp32_ble_tracker:327]: Starting scan... [09:48:37][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5 [09:49:57][W][bluetooth_proxy:217]: [0] [FA:71:15:24:9F:85] Connection already established [09:49:58][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5 [09:50:34][I][ota:113]: Boot seems successful, resetting boot loop counter. [09:50:34][D][esp32.preferences:113]: Saving 1 preferences to flash... [09:50:34][D][esp32.preferences:142]: Saving 1 preferences to flash: 0 cached, 1 written, 0 failed [09:51:18][W][bluetooth_proxy:217]: [0] [FA:71:15:24:9F:85] Connection already established [09:51:20][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5 [09:51:28][W][bluetooth_proxy:217]: [0] [FA:71:15:24:9F:85] Connection already established [09:51:29][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5 [09:51:34][W][bluetooth_proxy:217]: [0] [FA:71:15:24:9F:85] Connection already established [09:51:36][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5 [09:51:46][W][bluetooth_proxy:217]: [0] [FA:71:15:24:9F:85] Connection already established [09:51:48][W][bluetooth_proxy.connection:081]: [0] [FA:71:15:24:9F:85] Error reading char/descriptor at handle 0x24, status=5`

And I see this in the HA logs: (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Bluetooth GATT Error address=FA:71:15:24:9F:85 handle=36 error=5 description=Insufficient authentication (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Timeout waiting for connect response while connecting to FA:71:15:24:9F:85 after 20.0s, disconnect timed out: True, after 5.0s (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Connection not done for esp32-bluetooth-proxy-0464bc @ 192.168.1.221; current state is ConnectionState.INITIALIZED! (FA:71:15:24:9F:85) - FA:71:15:24:9F:85: Failed to connect: Error ESP_GATT_CONN_FAIL_ESTABLISH while connecting: Connection failed to establish: Interference/range; External Bluetooth adapter w/extension may help; Extension cables reduce USB 3 port interference

winstona commented 1 year ago

@cadwizzard are you using my override branch for the python-ember-mug lib in this output? I also remember seeing this error, I think that was when i went through and slowed down all of the bluetooth requests to see which was actually failing - leading to finding the DSK char having issues

cadwizzard commented 1 year ago

Hi @winstona - no. I thought you intended the message for @sopelj for testing, rather than everyone else who are the 'users'. Yes, I can edit config files, but when you mentioned kill the docker container, that assumes that people are a certain level to know how to do that. As I was unsure if rebooting the host would force the branch to be pulled, I didn't reboot the host. I keep that to a minimum as 98% of my devices are z wave and there are some issues with the 700 series controllers I wont bore you with, but generally only restart the host/HA when I have to :) I know of docker (I use toggledbits MSR engine) but I use HA supervised on Pi in what's technically an unsupported way (I think in docker though).

winstona commented 1 year ago

sorry about that @cadwizzard , that was intended to be directed at you (if you were up for testing it out or wanted to get something at least kind of working for now) - I understand the complications with restarting/etc due to zwave issues.

I saw some issues where a simple "restart" within HA settings was not enough to pull it in (or if it was it wasn't enough to update the branch, maybe it doesn't do a git pull/fetch before checking to see if the branch has updated and needed to use a commit hash instead), so I just resorted to killing/restarting the docker container when I was iterating on/testing this out.

You may be able to simply modify the manifest.json and do a restart from within HA (like you'd usually need to do for adding new components from HACS).

The alternative could be for @sopelj to merge my branch in (or make appropriate changes to remove DSK for now) and make a new beta release available, however afaik you'd still need to restart HA in one way or another to apply/load those changes.

cadwizzard commented 1 year ago

Thats ok @winstona I've actually never needed to modify manifest.json when adding HACS components (But I also have only been using HA for a year, after a decade with the MIOS/Vera/EZLO platform instead). I'll see what @sopelj next update is (maybe your branches will be in a beta?) and have a go at editing the configs after that if not. I do wonder if @sopelj has any plans in the pipeline to add the ability to track a second Ember mug. With Christmas coming up, there may be 2 in peoples households shortly ;)

smurf12345 commented 1 year ago

Hi All. I just found this awesome integration and looks like I'm running into the same issue.

I'm running the latest: HA: 2022.12.8 ESPHome: 2022.12.13 bluethooth proxy on an esp32 Integration/Component: 0.4.1-beta-2. Ember Mug v1 2022-12-29 08:45:54.010 INFO (MainThread) [custom_components.ember_mug.coordinator] Ember Mug ember-mug-a04de14cf35902fea05e8e322aec613d Setup 2022-12-29 08:45:54.011 DEBUG (MainThread) [custom_components.ember_mug.coordinator] Updating 2022-12-29 08:45:54.903 ERROR (MainThread) [custom_components.ember_mug.coordinator] Bluetooth GATT Error address=C0:38:70:C4:71:07 handle=36 error=5 description=Insufficient authentication 2022-12-29 08:45:54.907 DEBUG (MainThread) [custom_components.ember_mug] Finished fetching ember-mug-a04de14cf35902fea05e8e322aec613d data in 0.896 seconds (success: False) 2022-12-29 08:45:54.908 WARNING (MainThread) [homeassistant.config_entries] Config entry 'Ember Ceramic Mug' for ember_mug integration not ready yet: An error occurred updating mug: e=BleakError('Bluetooth GATT Error address=C0:38:70:C4:71:07 handle=36 error=5 description=Insufficient authentication'); Retrying in background 2022-12-29 08:45:59.911 INFO (MainThread) [custom_components.ember_mug.coordinator] Ember Mug ember-mug-a04de14cf35902fea05e8e322aec613d Setup 2022-12-29 08:45:59.912 DEBUG (MainThread) [custom_components.ember_mug.coordinator] Updating 2022-12-29 08:46:01.647 ERROR (MainThread) [custom_components.ember_mug.coordinator] Bluetooth GATT Error address=C0:38:70:C4:71:07 handle=36 error=5 description=Insufficient authentication 2022-12-29 08:46:01.648 DEBUG (MainThread) [custom_components.ember_mug] Finished fetching ember-mug-a04de14cf35902fea05e8e322aec613d data in 1.737 seconds (success: False)

sopelj commented 1 year ago

@smurf12345 Hi, Insufficient authentication seems to indicate it was never paired. Pairing is not currently possible via Bluetooth proxies, so you may need to bring the device next to the main bluetooth device for initial setup. Once paired, in theory it should be able to query via the proxy, but as stated I have yet to test.

@cadwizzard It may already work with multiple mugs. In theory it should use device IDs for the components, but it may also require some tweaks. I have no way of testing it, though. Please let me know if you do try it out. Whether it works or not, it would be nice to know.

cadwizzard commented 1 year ago

@sopelj Two mugs do work fine with the integration. "Lauras" mug is close to the HA install/USB BT adapter in the Pi and detects and works great... mine being near the BT proxy device doesnt when in that place still... but they are both there and reporting ok once I paired the second mug. Screenshot_20230104_092741_Home Assistant Screenshot_20230104_092732_Home Assistant

smurf12345 commented 1 year ago

@smurf12345 Hi, Insufficient authentication seems to indicate it was never paired. Pairing is not currently possible via Bluetooth proxies, so you may need to bring the device next to the main bluetooth device for initial setup. Once paired, in theory it should be able to query via the proxy, but as stated I have yet to test.

Thanks the info @sopelj. I think I was miss understanding the thread regarding bluetooth proxy and a fix in 0.4.1-beta-2. I popped a regular old usb bluetooth adapter into the HA server in the basement and pairs 100% but does not work when away from the usb bluetooth adapter and close to the proxy. Now I eagerly await Bluetooth proxy support, but from what I gather that is more of any issue on the ESPHome bluetooth proxy code that doesn't support pairing at the moment.

Keep up the awesome work!

austin202220 commented 1 year ago

I consider my self to be pretty proficient with esphome and the bluetooth proxies, I have multiple setup all over my property to pick up various BT devices that are out of range from my home assistant raspi pi. I am happy to help if you need any extra testing done. However, I suspect this is never going to work with the proxies due to a limitation imposed by the ember mug software which binds to one specific BT device at a time. Let me know if there is any testing I do to help out with this project

LCMilstein commented 1 year ago

Keeping this thread alive, I am having the same insufficient authentication issue. To be clear, though, for all non-ember bluetooth integrations, my understanding is that a proxy is sufficient. I have 2 ESP32 modules flashed with the ezbt resource from nabucasa. They're seen by HA and other BT devices are picked up by them. I understand this was originally for passive modes only, but that this should have changed in more recent releases. I can't test with BT plugged directly into HA as there's no support for the BT dongle on my hardware setup, which is why I'm relying heavily on the proxies. I don't have a deep enough understanding of what the differences are to help gain support for the setup, but I'd love to be able to integrate Ember via BT proxy like others in the thread, so if there are things I'm supposed to be trying to test out what you're building, please clarify/lmk and I'll follow the steps. Thanks for working on this!

MrKuenning commented 1 year ago

I've been trying to get it working with a proxy and always got an error saying it could not load the ember module. But today I just updated to 2023.2 and immediately it detected the mug and asked me if I wanted to add it. I took this as a great sign. But I'm now facing the authentication error. I tried resetting the mug and pairing it again. It shows up but still throws the error.

sopelj commented 1 year ago

@austin202220 Thanks! I appreciate the offer. I tried with one proxy here and I had the same issue on 2023.1. I have not yet tried on Home Assistant 2023.2, but I did not see any specific changes that might help, but it could. I created a release for the second beta of 0.5.0 which changes how it works a bit, but doesn't specifically fix anything related to the proxies. It will require more debugging. You can enable a very, very verbose mode on proxies in the ESPHome with:


esp32:
  framework:
    sdkconfig_options:
      CONFIG_BT_LOG_GATT_TRACE_LEVEL: VERBOSE

logger:
  level: VERY_VERBOSE

I'm going to try and test it more this week if I can find the time.

@LCMilstein Yes, the proxies do support "Active" Bluetooth calls, but the mug seems to require a paired connection before allowing queries and the proxies do not support pairing, hence the permission denied issue. The other integrations all seem to simply use passive BLE broadcasts and some "active" queries that do not require pairing. I'm hoping we can either find a workaround or that ESPHome adds the ability to pair.

@MrKuenning Do you only use a Proxy or o you have a Bluetooth Adapter? Because possibly the Mug simply succeeded at connecting to your main adapter, which works fine. The permission issue only seems to occur with the proxies.

winstona commented 1 year ago

@sopelj - have you tried removing the get_dsk call entirely? I haven't had a chance to look much at the recent updates, but some time ago it wasn't being used at all and iirc that was the only call that triggered pairing.

It won't be a perfect experience, but maybe partially working for some people will be better than not at all. I'm still running on my older hacked fork and using it daily with only ever having a bluetooth proxy connected so far. I have to put it in pairing mode for a few seconds every time HA or the bluetooth proxy restarts, but its better than 0.

sopelj commented 1 year ago

@sopelj - have you tried removing the get_dsk call entirely? I haven't had a chance to look much at the recent updates, but some time ago it wasn't being used at all and iirc that was the only call that triggered pairing.

It won't be a perfect experience, but maybe partially working for some people will be better than not at all. I'm still running on my older hacked fork and using it daily with only ever having a bluetooth proxy connected so far. I have to put it in pairing mode for a few seconds every time HA or the bluetooth proxy restarts, but its better than 0.

@winstona Oh really? Thanks! Did you get the permission denied error too before that? Do you have the Mug 1 or 2? I thought that DSK issue was related to the Mug 1. My goal with the DSK and UDSK was to try and allow for setting it up in Home Assistant without having to go through the App initially, but I have not yet managed to figure that out, so you're correct, it does not currently serve any purpose. So I could remove it or at the very least catch the error and prevent it from failing.

MrKuenning commented 1 year ago

@sopelj I only have a Proxy. My HASS is in a crawl space and has ZWAVE and ZigBee working from there, but I never added Bluetooth because I did not think it would reach very far. I am not trying to setup multiple proxies for roaming around the house. I just want one proxy to use as a remote HASS Bluetooth connection for various devices.

winstona commented 1 year ago

@sopelj - I have the Mug v1, however I didn't notice anything broken (except for pairing) with my v1, so I was assuming it was pretty much the same between v1 and v2. It's possible that there are some differences, but I'd think making DSK (and possibly UDSK) optional might be a good route to test out at least.

sopelj commented 1 year ago

Ok, in 0.5.0 beta 3, I removed the non-essential attributes (I might add an option to enable them later). I tested quickly with my Bluetooth Proxy, and it seemed to actually be able to read certain attributes from the ESP. Although, it would only connect once it had manually been placed in pairing mode and remains thus unless manually removed, but it is encouraging. I will try and do some more tests, but if anyone is willing to do some testing too, it would be much appreciated. Thanks! (Please keep in mind, there are a few changes in 0.5 and entities were renamed, so please check the changelog if you have issues)

MrKuenning commented 1 year ago

Can confirm! I updated to beta-3 and was able to pair my mug right away with my proxy.

For reference: I am only using a ESP32 Proxy, no local Bluetooth My mug is a few weeks old HW v10 and running the latest firmware 394

I did not even remove or unpair from my phone. I just put the mug in pairing mode and added on HASS and it was successful.

sopelj commented 1 year ago

Can confirm! I updated to beta-3 and was able to pair my mug right away with my proxy.

For reference: I am only using a ESP32 Proxy, no local Bluetooth My mug is a few weeks old HW v10 and running the latest firmware 394

I did not even remove or unpair from my phone. I just put the mug in pairing mode and added on HASS and it was successful.

That's awesome! Thanks for testing. No weird errors? Are you able to update values too? (This only works if the mug was configured with the mobile app)

MrKuenning commented 1 year ago

This is my first time using it via hass. Over the course of a day there was twice that the mug went unavailable. The first time I just removed it from hass and paired it again. The second time it happened, I had decided to make a cup of tea in it and I was watching the temp drop down to what I set it to. As soon as it reached the perfect temp it went unavailable. I looked at the proxy logs and it was saying connection failed. Rebooting hass did not bring the connection back. However just placing the mug in pairing mode immediately restored the connection.

The other symptom I ran into was Fahrenheit in Celsius seemed to be reversed. Meaning when I selected C it showed degrees in F and vice versa.

LCMilstein commented 1 year ago

Beta 3 also seems to work for me with the proxies!! UPDATE: HOWEVER, it keeps going unavailable when it's idle and requires me to re-pair (press and hold the button until it flashes blue and reload the integration).

sopelj commented 1 year ago

Hmm, that's a good start still. Thanks for testing. Do you only have on proxy or more than one? Do you have any errors in your logs?

That's strange. Is it outputing the wrong value or is it just the "select" for the unit that is wrong? There was an issue with the select that I fixed in 0.5.0-beta-4. Are you still having issues?

@LCMilstein Yay! Still, it's much better... :/ You shouldn't need to reload the integration, just put it in pairing mode and wait. Do you have any errors?

Does updating values work via proxies? Eg. setting the LED colour and target temp?

LCMilstein commented 1 year ago

@sopelj

This error originated from a custom integration.

Logger: custom_components.ember_mug.coordinator Source: custom_components/ember_mug/coordinator.py:85 Integration: Ember Mug (documentation, issues) First occurred: 5:29:26 PM (1 occurrences) Last logged: 5:29:26 PM

Update called, but no callback in []. Registering again.

MrKuenning commented 1 year ago

2023-02-03 18_04_59-NVIDIA GeForce Overlay DT

2023-02-03 18_03_48-NVIDIA GeForce Overlay DT

sopelj commented 1 year ago

Oh OK. Thanks! Yeah, I wonder if they might be arguing over it. Normally nothing can "pair" with a Proxy, but it does try to maintain a connection. I'm not sure how the proxies decide to switch. That's quite encouraging the LED works with the proxies. I was worried it might cause permission issues.

Oh, I'm sorry. There clearly are some issues with Fahrenheit, to be honest, I hadn't tried using it... Home Assistant does some magic behind the scenes and it appears to be backfiring here. There are multiple places the unit can be set. Where do you have it set to Fahrenheit? (in Home Assistant for your home, when you set up this integration, in the select in the Mug device?)

If the set_temperature call fails, you should see an actual error, either in the log or in the UI (a little popup at the bottom of the screen). That warning is unrelated. Do you have debug logging enabled? (I added a section to the readme to explain)

@MrKuenning Thanks for the info. You can set the temp in Fahrenheit or just Celsius? That's weird about the liquid level, under the attributes is the "raw_liquid_level" 30? Maybe it wasn't updated. Becoming unavailable should only happen if there are Bluetooth errors. Do you have some in your logs?

sopelj commented 1 year ago

On the Fahrenheit front, I think the issue is that the mug attribute doesn't seem to actually do anything. So, I removed my conversion logic, in beta-5, and Home Assistant should (in theory) do the conversion automatically if you set it to Fahrenheit.

MrKuenning commented 1 year ago

@sopelj I would be happy to share logs. Which do you want? HASS general logs? Proxy Console? Other?

My process as of right now is to put the mug into pairing mode every time I make a drink. It generally stays connected until it powers off at which time it will not reconnect until it's put into pairing mode again. Sometimes it drops off even while the mug is still on.

sopelj commented 1 year ago

@sopelj I would be happy to share logs. Which do you want? HASS general logs? Proxy Console? Other?

My process as of right now is to put the mug into pairing mode every time I make a drink. It generally stays connected until it powers off at which time it will not reconnect until it's put into pairing mode again. Sometimes it drops off even while the mug is still on.

Oh OK. That's unfortunate. This is on the latest beta? What is your setup of proxies? Do you have just one or many? Bluetooth adapter as well? Both the Home Assistant log and ESPHome one would be helpful if possible. I added a section for debugging if that helps: https://github.com/sopelj/hass-ember-mug-component#debugging Thanks!