search

canonical / layer-basic

Base layer for building charms using the reactive pattern
Apache License 2.0
11 stars 43 forks source link

Deployment fails when use_venv == false and series == bionic #168

Closed ittner closed 4 years ago

ittner commented 4 years ago

A charm using the current layer-basic with use_venv: false in layer.yaml will fail deployment on bionic; An automatically-added dependency will cause pip3 to attempt to update PyYAML to the version available in the wheelhouse (5.2), trying to replace version 3.12 automatically installed from apt, which will fail.

It seems the --force-reinstall from commit f8629dbe4eba480677eb613f218f8a0a384709e4 caused a regression for the fixes committed in issues #146 and #147.

A minimal layer.yaml to trigger the issue is:

includes:
  - layer:basic
options:
  basic:
    use_venv: false
    python_packages: [ ]
    packages: [ ]

There is a minimal charm demo'ing this issue in: https://git.launchpad.net/~aittner/+git/no-venv-poc

juju debug-log

unit-no-venv-poc-0: 01:11:42 DEBUG unit.no-venv-poc/0.install Looking in links: wheelhouse
unit-no-venv-poc-0: 01:11:42 DEBUG unit.no-venv-poc/0.install Collecting netaddr
unit-no-venv-poc-0: 01:11:43 DEBUG unit.no-venv-poc/0.install Collecting Jinja2
unit-no-venv-poc-0: 01:11:44 DEBUG unit.no-venv-poc/0.install Collecting MarkupSafe
unit-no-venv-poc-0: 01:11:45 DEBUG unit.no-venv-poc/0.install Collecting charms.reactive
unit-no-venv-poc-0: 01:11:46 DEBUG unit.no-venv-poc/0.install Collecting wheel
unit-no-venv-poc-0: 01:11:47 DEBUG unit.no-venv-poc/0.install Collecting Tempita
unit-no-venv-poc-0: 01:11:48 DEBUG unit.no-venv-poc/0.install Collecting pyaml
unit-no-venv-poc-0: 01:11:48 DEBUG unit.no-venv-poc/0.install Collecting charmhelpers
unit-no-venv-poc-0: 01:11:49 DEBUG unit.no-venv-poc/0.install Collecting six
unit-no-venv-poc-0: 01:11:50 DEBUG unit.no-venv-poc/0.install Collecting PyYAML
unit-no-venv-poc-0: 01:11:52 DEBUG unit.no-venv-poc/0.install Installing collected packages: netaddr, MarkupSafe, Jinja2, PyYAML, pyaml, six, Tempita, charmhelpers, charms.reactive, wheel
unit-no-venv-poc-0: 01:11:52 DEBUG unit.no-venv-poc/0.install   Running setup.py install for netaddr: started
unit-no-venv-poc-0: 01:11:53 DEBUG unit.no-venv-poc/0.install     Running setup.py install for netaddr: finished with status 'done'
unit-no-venv-poc-0: 01:11:53 DEBUG unit.no-venv-poc/0.install   Found existing installation: MarkupSafe 1.0
unit-no-venv-poc-0: 01:11:53 DEBUG unit.no-venv-poc/0.install     Uninstalling MarkupSafe-1.0:
unit-no-venv-poc-0: 01:11:53 DEBUG unit.no-venv-poc/0.install       Successfully uninstalled MarkupSafe-1.0
unit-no-venv-poc-0: 01:11:53 DEBUG unit.no-venv-poc/0.install   Running setup.py install for MarkupSafe: started
unit-no-venv-poc-0: 01:11:54 DEBUG unit.no-venv-poc/0.install     Running setup.py install for MarkupSafe: finished with status 'done'
unit-no-venv-poc-0: 01:11:54 DEBUG unit.no-venv-poc/0.install   Found existing installation: Jinja2 2.10
unit-no-venv-poc-0: 01:11:54 DEBUG unit.no-venv-poc/0.install     Uninstalling Jinja2-2.10:
unit-no-venv-poc-0: 01:11:54 DEBUG unit.no-venv-poc/0.install       Successfully uninstalled Jinja2-2.10
unit-no-venv-poc-0: 01:11:54 DEBUG unit.no-venv-poc/0.install   Running setup.py install for Jinja2: started
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install     Running setup.py install for Jinja2: finished with status 'done'
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install   Found existing installation: PyYAML 3.12
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install Traceback (most recent call last):
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install   File "/var/lib/juju/agents/unit-no-venv-poc-0/charm/hooks/install", line 8, in <module>
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install     basic.bootstrap_charm_deps()
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install   File "lib/charms/layer/basic.py", line 180, in bootstrap_charm_deps
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install     '--no-cache-dir', '-f', 'wheelhouse'] + list(pkgs))
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install     raise CalledProcessError(retcode, cmd)
unit-no-venv-poc-0: 01:11:55 DEBUG unit.no-venv-poc/0.install subprocess.CalledProcessError: Command '['pip3', 'install', '-U', '--force-reinstall', '--no-index', '--no-cache-dir', '-f', 'wheelhouse', 'netaddr', 'Jinja2', 'MarkupSafe', 'charms.reactive', 'wheel', 'Tempita', 'pyaml', 'charmhelpers', 'six', 'PyYAML']' returned non-zero exit status 1.
unit-no-venv-poc-0: 01:11:56 ERROR juju.worker.uniter.operation hook "install" failed: exit status 1
dreibh commented 4 years ago

I can confirm the issue for my charms as well. The problem of trying to install PyYAML with pip3 also occurs for deployment on xenial.