Consider using python3Full for home-assistant

[justinas]

Issue description

Certain custom Home Assistant components, such as ble_monitor which I'm attempting to package require Python built with Bluetooth support (a.k.a. bluezSupport) which is present in the "full" flavor of Python, but not the default one in nixpkgs (python3).

I would like to suggest using pythonFull for home-assistant instead. Considering that home-assistant is already a large derivation with a closure size of around 800MB, this should hopefully not have too large of an impact:

$ nix path-info -Sh $(nix-build -A python312 '<nixpkgs>')
/nix/store/9yj3k8zvnb4pkbyjaxmb1wkn6ayrxvxy-python3-3.12.4       165.6M
$ nix path-info -Sh $(nix-build -A python312Full '<nixpkgs>')
/nix/store/52fcvpkl4cy8gz6qjxm0crfx2m4xgrdj-python3-3.12.4       187.0M

While one can always home-assistant.override { python312 = python312Full }, this results in having to rebuild all of H-A's Python dependencies, which may take up to multiple hours on modest hardware. My understanding is that if home-assistant used python312Full by default, Hydra would cache these and local rebuilds would not be necessary.

[chewblacka]

You might want to tag the maintainers of home-assistant

[justinas]

You might want to tag the maintainers of home-assistant

Ah, I meant to! :slightly_smiling_face: Thanks

/cc @fabaff @Mic92

[mweinelt]

Wondering if we should just override bluezSupport, so we don't pull in x11 dependencies onto headless systems. I'm also not fond of the increased build and review cost of an overriden python package.

I think the best course of action is to understand why bluezSupport is not available in the default interpreter, and see if we can work around that.

[mweinelt]

The dependency on bluez pulls in half the world, that in turn depends on python3, which makes including it by default a bit annoying:

• asciidoc
• audit
• autoPatchelfHook
• gtk-doc
• itstool
• libxml2
• libxslt
• meson
• ninja
• gobject-introspection
• re2c
• systemd/udev
• xorg

[dotlambda]

Certain custom Home Assistant components, such as ble_monitor which I'm attempting to package require Python built with Bluetooth support

Is that because https://github.com/ukBaz/python-btsocket requires it? We have that packaged and so far nobody complained about it not being supported.

[dotlambda]

I guess it actually comes from https://github.com/frawau/aioblescan/blob/ac9d567af997a1e9d9f9a18bc23b3295825fa823/aioblescan/aioblescan.py#L1691. Could we replace that with a constant as in https://github.com/ukBaz/python-btsocket/blob/e1f8d0c5d3166295d91ab1f1e9c0ad7a0559ca3b/btsocket/btmgmt_socket.py#L7?

[justinas]

I guess it actually comes from https://github.com/frawau/aioblescan/blob/ac9d567af997a1e9d9f9a18bc23b3295825fa823/aioblescan/aioblescan.py#L1691. Could we replace that with a constant as in https://github.com/ukBaz/python-btsocket/blob/e1f8d0c5d3166295d91ab1f1e9c0ad7a0559ca3b/btsocket/btmgmt_socket.py#L7?

Yes, that is the problematic line.

2024-09-09 14:31:18.368 ERROR (Thread-2) [root] Uncaught thread exception
Traceback (most recent call last):
  File "/nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/var/lib/hass/custom_components/ble_monitor/__init__.py", line 630, in run
    mysocket[hci] = aiobs.create_bt_socket(hci)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/j5l7w60m47xxcz49z738rdjx6hkxw1xs-python3.12-aioblescan-0.2.14/lib/python3.12/site-packages/aioblescan/aioblescan.py", line 1691, in>
    family=socket.AF_BLUETOOTH, type=socket.SOCK_RAW, proto=socket.BTPROTO_HCI
           ^^^^^^^^^^^^^^^^^^^
AttributeError: module 'socket' has no attribute 'AF_BLUETOOTH'

I would be quite surprised if this is the only reason we fail when Bluetooth support is not compiled-in, but I can try to patch this out and see if that works :man_shrugging: To be honest, I have close to zero knowledge in how Python / Bluetooth / Bluez interact, and why other software (or even other Home Assistant components) use Bluetooth just fine. I guess those might be using the D-Bus API instead of creating low-level sockets?

[justinas]

An update on the above: I patched out the constants that aioblescan was hard-erroring on:

diff --git a/aioblescan/aioblescan.py b/aioblescan/aioblescan.py
index 3845b91..b994eb4 100644
--- a/aioblescan/aioblescan.py
+++ b/aioblescan/aioblescan.py
@@ -28,6 +28,10 @@
 import socket, platform, asyncio
 from struct import pack, unpack, calcsize

+AF_BLUETOOTH = 31
+BTPROTO_HCI = 1
+SOL_HCI = 0
+HCI_FILTER = 2

 # A little bit of HCI
 HCI_COMMAND = 0x01
@@ -1688,13 +1692,13 @@ def create_bt_socket(interface=None):
     sock = None
     try:
         sock = socket.socket(
-            family=socket.AF_BLUETOOTH, type=socket.SOCK_RAW, proto=socket.BTPROTO_HCI
+            family=AF_BLUETOOTH, type=socket.SOCK_RAW, proto=BTPROTO_HCI
         )
         sock.setblocking(False)
         if platform.system() == "Linux":
             sock.setsockopt(
-                socket.SOL_HCI,
-                socket.HCI_FILTER,
+                SOL_HCI,
+                HCI_FILTER,
                 pack("IIIh2x", 0xFFFFFFFF, 0xFFFFFFFF, 0xFFFFFFFF, 0),
             )  # type mask, event mask, event mask, opcode
             if not interface:
@@ -1712,8 +1716,8 @@ def create_bt_socket(interface=None):
             import ctypes, ctypes.util

             sock.setsockopt(
-                socket.SOL_HCI,
-                socket.HCI_FILTER,
+                SOL_HCI,
+                HCI_FILTER,
                 pack("IQ", 0x00000000, 0x2000000000000000),
             )  # type mask, event mask
             if not interface:
@@ -1729,7 +1733,7 @@ def create_bt_socket(interface=None):

             adr = SockaddrHci(
                 ctypes.sizeof(SockaddrHci),
-                socket.AF_BLUETOOTH,
+                AF_BLUETOOTH,
                 (interface + "hci").ljust(32, "\0").encode("utf-8"),
             )
             if (

but no luck, it is now probably hitting the root problem of bluetooth support not being available?

2024-09-23 14:40:55.495 ERROR (Thread-2) [custom_components.ble_monitor] HCIdump thread: OS error (hci0): [Errno None] error while attempting to bind on interface 0: None

[mweinelt]

@AndersonTorres Any chance we can get a bluezMinimal derivation, that does not depend on Python, so we can use it to build the standard python interpreter with bluetooth support?

[AndersonTorres]

Yep! Just send the patches! Right now I am adopting and fixing httplib update: https://github.com/NixOS/nixpkgs/pull/343963 But I can look at Bluez more carefully asap.

Edit:

https://github.com/NixOS/nixpkgs/pull/344094