Unable to load recent version of pyads library

[henning-meinhardt]

Problem/Motivation

Using v0.7.1 of this addon I'm unable to load a recent version of the pyads library

init_commands: []
python_packages:
  - pyads==3.3.7
system_packages:
  - make

When omitting the version number (i.e. specifying just pyads instead of pyads==3.3.7) the library is successfully loaded, however using a quite old version 3.2.(some) which misses a function added in 3.3.0 that I rely on. I tried different versions starting with 3.3.0 up to 3.3.9 (latest) but none of them can be loaded. The problem is basically that a shared system library that should be installed with the pyads library can not be found. The addon starts with an error message saying that no wheel could be built and so installation falls back to a legacy method. Later, when importing the library in an app, another error is logged saying that the system library is not available.

Unfortunately I'm not a python guy so I can't see from the logs whether the problem is caused by this addon or the pyads library. So I'll address the same question also to the author of the pyads library. Maybe there is a way to manually build the shared library using the init_commands option. According to https://pyads.readthedocs.io/en/latest/installation.html the system library may be manually build using

cd adslib
make
sudo make install

But I don't know how to leverage this in the init_commands

Expected behavior

It should be possible to use a recent version of the pyads library (ideally 3.3.9) with this addon.

Actual behavior

Restarting the addon gives the following error in the logs:

-----------------------------------------------------------
 Add-on: AppDaemon 4
 Python Apps and Dashboard using AppDaemon 4.x for Home Assistant
-----------------------------------------------------------
 Add-on version: 0.7.1
 You are running the latest version of this add-on.
 System: Home Assistant OS 7.0  (amd64 / qemux86-64)
 Home Assistant Core: 2021.12.7
 Home Assistant Supervisor: 2021.12.2
-----------------------------------------------------------
 Please, share the above information when looking for help
 or support in, e.g., GitHub, forums or the Discord chat.
-----------------------------------------------------------
[cont-init.d] 00-banner.sh: exited 0.
[cont-init.d] 01-log-level.sh: executing... 
[cont-init.d] 01-log-level.sh: exited 0.
[cont-init.d] appdaemon.sh: executing... 
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
v3.14.3-71-gffbd3325d6 [https://dl-cdn.alpinelinux.org/alpine/v3.14/main]
v3.14.3-73-g89aa18397a [https://dl-cdn.alpinelinux.org/alpine/v3.14/community]
OK: 14946 distinct packages available
(1/1) Installing make (4.3-r0)
Executing busybox-1.33.1-r3.trigger
OK: 87 MiB in 65 packages
Looking in links: https://wheels.home-assistant.io/alpine-3.13/amd64/
Collecting pyads==3.3.7
  Downloading pyads-3.3.7.tar.gz (318 kB)
Building wheels for collected packages: pyads
  Building wheel for pyads (setup.py): started
  Building wheel for pyads (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-bhbr62f2/pyads_8b339d34d91543db949f2707c1a40f0d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-bhbr62f2/pyads_8b339d34d91543db949f2707c1a40f0d/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-bvhvko4l
       cwd: /tmp/pip-install-bhbr62f2/pyads_8b339d34d91543db949f2707c1a40f0d/
  Complete output (21 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-bhbr62f2/pyads_8b339d34d91543db949f2707c1a40f0d/setup.py", line 152, in <module>
      setup(
    File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.9/distutils/core.py", line 134, in setup
      ok = dist.parse_command_line()
    File "/usr/lib/python3.9/distutils/dist.py", line 483, in parse_command_line
      args = self._parse_command_opts(parser, args)
    File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 916, in _parse_command_opts
      nargs = _Distribution._parse_command_opts(self, parser, args)
    File "/usr/lib/python3.9/distutils/dist.py", line 539, in _parse_command_opts
      cmd_class = self.get_command_class(command)
    File "/usr/lib/python3.9/site-packages/setuptools/dist.py", line 755, in get_command_class
      self.cmdclass[command] = cmdclass = ep.load()
    File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2450, in load
      return self.resolve()
    File "/usr/lib/python3.9/site-packages/pkg_resources/__init__.py", line 2456, in resolve
      module = __import__(self.module_name, fromlist=['__name__'], level=0)
  ModuleNotFoundError: No module named 'wheel.bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for pyads
  Running setup.py clean for pyads
Failed to build pyads
Installing collected packages: pyads
    Running setup.py install for pyads: started
    Running setup.py install for pyads: finished with status 'done'
  DEPRECATION: pyads was installed using the legacy 'setup.py install' method, because a wheel could not be built for it. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Successfully installed pyads-3.3.7
[cont-init.d] appdaemon.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[23:19:33] INFO: Starting AppDaemon...

Later when an app that imports the library is started, the log shows an other error:

2021-12-29 23:19:35.751312 INFO AppDaemon: Loading App Module: /config/appdaemon/apps/ads.py
2021-12-29 23:19:35.756630 WARNING Error: ------------------------------------------------------------
2021-12-29 23:19:35.756818 WARNING Error: Unexpected error loading module: /config/appdaemon/apps/ads.py:
2021-12-29 23:19:35.756925 WARNING Error: ------------------------------------------------------------
2021-12-29 23:19:35.759069 WARNING Error: Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/appdaemon/app_management.py", line 970, in check_app_updates
    await utils.run_in_executor(self, self.read_app, mod["name"], mod["reload"])
  File "/usr/lib/python3.9/site-packages/appdaemon/utils.py", line 308, in run_in_executor
    response = future.result()
  File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/lib/python3.9/site-packages/appdaemon/app_management.py", line 761, in read_app
    self.modules[module_name] = importlib.import_module(module_name)
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 855, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/config/appdaemon/apps/ads.py", line 2, in <module>
    import pyads
  File "/usr/lib/python3.9/site-packages/pyads/__init__.py", line 10, in <module>
    from .ads import (
  File "/usr/lib/python3.9/site-packages/pyads/ads.py", line 57, in <module>
    from .pyads_ex import (
  File "/usr/lib/python3.9/site-packages/pyads/pyads_ex.py", line 88, in <module>
    _adsDLL = ctypes.CDLL(adslib)
  File "/usr/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: Error loading shared library adslib.so: No such file or directory

Steps to reproduce

Use the following addon configuration

init_commands: []
python_packages:
  - pyads==3.3.7
system_packages:
  - make

and create an app that imports the library, e.g.:

import pyads
class ADS(hass.Hass):
  def initialize(self):
    ... anything here

Proposed changes

It would be nice to have a more detailed documentation of how a specific version of a library may be referenced (using ==<version>) - I had to google several sites until I found that by accident. Also, how exactly is init_commands meant to be used? What's the purpose of this, when is it executed/applied?

[stlehmann]

It seems the wheel package is missing. It is needed to generate a wheel from the source distribution. I propose you try adding wheel to the python_packages section of your addon configuration.

[henning-meinhardt]

Just tried that but to no avail. Adding wheel just produces the extra output

Looking in links: https://wheels.home-assistant.io/alpine-3.13/amd64/
Requirement already satisfied: wheel in /usr/lib/python3.9/site-packages (0.37.0)

The error remains:

ModuleNotFoundError: No module named 'wheel.bdist_wheel'

Also tried to add both, wheel as well as setuptools packages as proposed in some stackoverflow issue regarding another lib with a similar problem. Same result.

[frenck]

Oh man, not related, but your debug log shows a major stupid error/bug... 🤒 Alpine 3.13 wheels... we are on 3.14 already. Fixing that asap.

[github-actions[bot]]

There hasn't been any activity on this issue recently, so we clean up some of the older and inactive issues. Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thanks!