search

canonical / layer-filebeat

Filebeat is a lightweight log shipper. This is the source for the filebeat charm in the Juju charm store.
Other
3 stars 27 forks source link

Charm install broken on focal #112

Closed samuelallan72 closed 3 months ago

samuelallan72 commented 3 months ago

With a new build on main at the time of writing, built with charmcraft 2.6.0.

On focal, deploying the charm crashes:

unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install Successfully built MarkupSafe wheel
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install Installing collected packages: MarkupSafe, wheel
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install   Attempting uninstall: wheel
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install     Found existing installation: wheel 0.34.2
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install     Uninstalling wheel-0.34.2:
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install       Successfully uninstalled wheel-0.34.2
unit-filebeat-0: 14:28:41 DEBUG unit.filebeat/0.install Successfully installed MarkupSafe-2.0.1 wheel-0.33.6
unit-filebeat-0: 14:28:43 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 14:28:43 DEBUG unit.filebeat/0.install Processing ./wheelhouse/Jinja2-3.0.3.tar.gz
unit-filebeat-0: 14:28:43 DEBUG unit.filebeat/0.install Processing ./wheelhouse/charmhelpers-1.2.1.tar.gz
unit-filebeat-0: 14:28:45 DEBUG unit.filebeat/0.install Processing ./wheelhouse/sniffio-1.2.0.tar.gz
unit-filebeat-0: 14:28:45 DEBUG unit.filebeat/0.install   Installing build dependencies: started
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install   Installing build dependencies: finished with status 'done'
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install   Getting requirements to build wheel: started
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install   Getting requirements to build wheel: finished with status 'done'
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install     Preparing wheel metadata: started
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install     Preparing wheel metadata: finished with status 'done'
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install Processing ./wheelhouse/typing_extensions-4.1.1.tar.gz
unit-filebeat-0: 14:28:49 DEBUG unit.filebeat/0.install   Installing build dependencies: started
unit-filebeat-0: 14:28:50 DEBUG unit.filebeat/0.install   Installing build dependencies: finished with status 'error'
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   ERROR: Command errored out with exit status 1:
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install    command: /var/lib/juju/agents/unit-filebeat-0/.venv/bin/python /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-fqcjn89i/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- 'flit_core >=3.4,<4'
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install        cwd: None
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   Complete output (3 lines):
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   Looking in links: wheelhouse
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   ERROR: Could not find a version that satisfies the requirement flit_core<4,>=3.4 (from versions: none)
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   ERROR: No matching distribution found for flit_core<4,>=3.4
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install   ----------------------------------------
unit-filebeat-0: 14:28:50 WARNING unit.filebeat/0.install ERROR: Command errored out with exit status 1: /var/lib/juju/agents/unit-filebeat-0/.venv/bin/python /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-fqcjn89i/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- 'flit_core >=3.4,<4' Check the logs for full command output.
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install Traceback (most recent call last):
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install   File "/var/lib/juju/agents/unit-filebeat-0/charm/hooks/install", line 8, in <module>
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install     basic.bootstrap_charm_deps()
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install   File "/var/lib/juju/agents/unit-filebeat-0/charm/lib/charms/layer/basic.py", line 224, in bootstrap_charm_deps
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install     check_call([pip, 'install', '-U', reinstall_flag, '--no-index',
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install     raise CalledProcessError(retcode, cmd)
unit-filebeat-0: 14:28:51 WARNING unit.filebeat/0.install subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-filebeat-0/.venv/bin/pip', 'install', '-U', '--force-reinstall', '--no-index', '--no-cache-dir', '-f', 'wheelhouse', 'Jinja2==3.0.3', 'charmhelpers==1.2.1', 'sniffio==1.2.0', 'typing-extensions==4.1.1', 'netaddr==0.7.19', 'contextvars==2.4', 'pyaml==21.10.1', 'pbr==6.0.0', 'PyYAML==5.3.1', 'charms.reactive==1.5.2', 'Cython==0.29.37', 'immutables==0.19']' returned non-zero exit status 1.
unit-filebeat-0: 14:28:51 ERROR juju.worker.uniter.operation hook "install" (via explicit, bespoke hook script) failed: exit status 1
unit-filebeat-0: 14:28:51 INFO juju.worker.uniter awaiting error resolution for "install" hook
samuelallan72 commented 3 months ago

If I build the charm on focal, and run on focal, I get a different error:

unit-filebeat-0: 16:39:13 DEBUG unit.filebeat/0.install Unpacking virtualenv (20.0.17-1ubuntu0.4) ...
unit-filebeat-0: 16:39:13 DEBUG unit.filebeat/0.install Setting up virtualenv (20.0.17-1ubuntu0.4) ...
unit-filebeat-0: 16:39:15 DEBUG unit.filebeat/0.install created virtual environment CPython3.8.10.final.0-64 in 198ms
unit-filebeat-0: 16:39:15 DEBUG unit.filebeat/0.install   creator CPython3Posix(dest=/var/lib/juju/agents/unit-filebeat-0/.venv, clear=False, global=False)
unit-filebeat-0: 16:39:15 DEBUG unit.filebeat/0.install   seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, pkg_resources=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1.debian.1)
unit-filebeat-0: 16:39:15 DEBUG unit.filebeat/0.install   activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
unit-filebeat-0: 16:39:17 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 16:39:17 DEBUG unit.filebeat/0.install Requirement already up-to-date: pip in /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages (20.0.2)
unit-filebeat-0: 16:39:19 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 16:39:19 DEBUG unit.filebeat/0.install Requirement already up-to-date: setuptools in /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages (44.0.0)
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Processing ./wheelhouse/setuptools_scm-1.17.0.tar.gz
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Building wheels for collected packages: setuptools-scm
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install   Building wheel for setuptools-scm (setup.py): started
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install   Building wheel for setuptools-scm (setup.py): finished with status 'done'
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install   Created wheel for setuptools-scm: filename=setuptools_scm-1.17.0-py2.py3-none-any.whl size=17696 sha256=bad9968c3faa3b40e2df10d290626f29482d5a99c2ec1a134194e1516c5b2e93
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install   Stored in directory: /root/.cache/pip/wheels/1d/5c/86/5510ebd26134e3ba0c566d7685b047ea7b47a592b67f7c4be1
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Successfully built setuptools-scm
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Installing collected packages: setuptools-scm
unit-filebeat-0: 16:39:20 DEBUG unit.filebeat/0.install Successfully installed setuptools-scm-1.17.0
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install Processing ./wheelhouse/wheel-0.33.6.tar.gz
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install Processing ./wheelhouse/MarkupSafe-2.0.1.tar.gz
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install Building wheels for collected packages: wheel, MarkupSafe
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install   Building wheel for wheel (setup.py): started
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install   Building wheel for wheel (setup.py): finished with status 'done'
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install   Created wheel for wheel: filename=wheel-0.33.6-py2.py3-none-any.whl size=21555 sha256=064787397c16d6ddfc105e55cf591a2b5c174ad925596d7b4da71b1def96fc57
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install   Stored in directory: /tmp/pip-ephem-wheel-cache-afahdt1b/wheels/ac/c6/e7/97b114da18f5e79e1374909494e3d9b6d7caedb00e1869ab21
unit-filebeat-0: 16:39:22 DEBUG unit.filebeat/0.install   Building wheel for MarkupSafe (setup.py): started
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install   Building wheel for MarkupSafe (setup.py): finished with status 'done'
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install   Created wheel for MarkupSafe: filename=MarkupSafe-2.0.1-cp38-cp38-linux_x86_64.whl size=30806 sha256=fd4a11b7bc1d145a6ac7b574900cbe297ff4c6f6d5e49f75bafc8143312803cc
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install   Stored in directory: /tmp/pip-ephem-wheel-cache-afahdt1b/wheels/3a/39/26/49de15c85270c90a2aeec02fb064790a9dbe8e0c52d7d3e363
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install Successfully built wheel MarkupSafe
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install Installing collected packages: wheel, MarkupSafe
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install   Attempting uninstall: wheel
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install     Found existing installation: wheel 0.34.2
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install     Uninstalling wheel-0.34.2:
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install       Successfully uninstalled wheel-0.34.2
unit-filebeat-0: 16:39:23 DEBUG unit.filebeat/0.install Successfully installed MarkupSafe-2.0.1 wheel-0.33.6
unit-filebeat-0: 16:39:24 DEBUG unit.filebeat/0.install Looking in links: wheelhouse
unit-filebeat-0: 16:39:24 DEBUG unit.filebeat/0.install Processing ./wheelhouse/pyaml-21.10.1.tar.gz
unit-filebeat-0: 16:39:25 DEBUG unit.filebeat/0.install Processing ./wheelhouse/Cython-0.29.37.tar.gz
unit-filebeat-0: 16:39:31 DEBUG unit.filebeat/0.install Processing ./wheelhouse/charmhelpers-1.2.1.tar.gz
unit-filebeat-0: 16:39:34 DEBUG unit.filebeat/0.install Processing ./wheelhouse/sniffio-1.2.0.tar.gz
unit-filebeat-0: 16:39:34 DEBUG unit.filebeat/0.install   Installing build dependencies: started
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install   Installing build dependencies: finished with status 'done'
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install   Getting requirements to build wheel: started
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install   Getting requirements to build wheel: finished with status 'done'
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install     Preparing wheel metadata: started
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install     Preparing wheel metadata: finished with status 'done'
unit-filebeat-0: 16:39:37 DEBUG unit.filebeat/0.install Processing ./wheelhouse/pbr-6.0.0.tar.gz
unit-filebeat-0: 16:39:38 DEBUG unit.filebeat/0.install Processing ./wheelhouse/charms.reactive-1.5.2.tar.gz
unit-filebeat-0: 16:39:38 DEBUG unit.filebeat/0.install Processing ./wheelhouse/contextvars-2.4.tar.gz
unit-filebeat-0: 16:39:38 DEBUG unit.filebeat/0.install Processing ./wheelhouse/PyYAML-5.3.1.tar.gz
unit-filebeat-0: 16:39:39 DEBUG unit.filebeat/0.install Processing ./wheelhouse/netaddr-0.7.19.tar.gz
unit-filebeat-0: 16:39:39 DEBUG unit.filebeat/0.install Processing ./wheelhouse/immutables-0.19.tar.gz
unit-filebeat-0: 16:39:39 DEBUG unit.filebeat/0.install   Installing build dependencies: started
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   ERROR: Command errored out with exit status 1:
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install    command: /var/lib/juju/agents/unit-filebeat-0/.venv/bin/python /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-op2jgdho/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- 'setuptools>=42' wheel
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install        cwd: None
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   Complete output (3 lines):
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   Looking in links: wheelhouse
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   ERROR: Could not find a version that satisfies the requirement setuptools>=42 (from versions: 41.6.0)
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   ERROR: No matching distribution found for setuptools>=42
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install   ----------------------------------------
unit-filebeat-0: 16:39:40 DEBUG unit.filebeat/0.install   Installing build dependencies: finished with status 'error'
unit-filebeat-0: 16:39:40 WARNING unit.filebeat/0.install ERROR: Command errored out with exit status 1: /var/lib/juju/agents/unit-filebeat-0/.venv/bin/python /var/lib/juju/agents/unit-filebeat-0/.venv/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-op2jgdho/overlay --no-warn-script-location --no-binary :none: --only-binary :none: --no-index --find-links wheelhouse -- 'setuptools>=42' wheel Check the logs for full command output.
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install Traceback (most recent call last):
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install   File "/var/lib/juju/agents/unit-filebeat-0/charm/hooks/install", line 8, in <module>
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install     basic.bootstrap_charm_deps()
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install   File "/var/lib/juju/agents/unit-filebeat-0/charm/lib/charms/layer/basic.py", line 224, in bootstrap_charm_deps
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install     check_call([pip, 'install', '-U', reinstall_flag, '--no-index',
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install     raise CalledProcessError(retcode, cmd)
unit-filebeat-0: 16:39:41 WARNING unit.filebeat/0.install subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-filebeat-0/.venv/bin/pip', 'install', '-U', '--force-reinstall', '--no-index', '--no-cache-dir', '-f', 'wheelhouse', 'pyaml==21.10.1', 'Cython==0.29.37', 'charmhelpers==1.2.1', 'sniffio==1.2.0', 'pbr==6.0.0', 'charms.reactive==1.5.2', 'contextvars==2.4', 'PyYAML==5.3.1', 'netaddr==0.7.19', 'immutables==0.19', 'Jinja2==3.0.3', 'typing-extensions==4.1.1']' returned non-zero exit status 1.

This is the same error that was reported in https://github.com/canonical/layer-basic/issues/223 , but that should have been fixed.

samuelallan72 commented 3 months ago

So far I've been able to fix this by adding the following lines to wheelhouse.txt (I had to create the file; we don't have a local wheelhouse.txt yet - we just rely on the one from layer-basic):

immutables<0.16
flit_core<4.0.0

And also building the charm on focal (not sure if this makes a difference yet though). We may or may not need to separate the builds_on and runs_on.

I'm not sure if this is the best fix though (I'm also confused about flit_core, since it's in wheelhouse.txt already for python >= 3.8). And I don't know if we want to fix it here in this charm, or in layer-basic (see conversation on https://github.com/canonical/layer-basic/issues/223 ).