Reactive charms built with charm version 3.x does not install on bionic (Python >=3.7 is required, 3.6.9 is present)

The original issue was reported in charmcraft repository, but probably the issue is that reactive losing compatibility with bionic

Building a dummy reactive charm with charm 3.x results in an error in install hook. Example charmcraft.yaml:

type: charm
parts:
  charm:
    source: src
    plugin: reactive
    build-snaps: 
      - charm
bases:
    - build-on:
        - name: ubuntu
          channel: "22.04"
          architectures: ["amd64"]
      run-on:
        - name: ubuntu
          channel: "22.04"
          architectures:
              - amd64
        - name: ubuntu
          channel: "20.04"
          architectures:
              - amd64
        - name: ubuntu
          channel: "18.04"
          architectures:
              - amd64

Installation in bionic environment results in such error log

022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60 pip requires Python '>=3.7' but the running Python is 3.6.9
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60 Traceback (most recent call last):
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60   File "/var/lib/juju/agents/unit-graylog-0/charm/hooks/install", line 8, in <module>
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60     basic.bootstrap_charm_deps()
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60   File "lib/charms/layer/basic.py", line 202, in bootstrap_charm_deps
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60     _update_if_newer(pip, pre_install_pkgs)
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60   File "lib/charms/layer/basic.py", line 367, in _update_if_newer
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60     pkg], env=_get_subprocess_env())
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60     raise CalledProcessError(retcode, cmd)
2022-10-06 10:22:33 WARNING unit.graylog/0.install logger.go:60 subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-graylog-0/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status 1.

The complete error log is available here

For some reason, downgrading charm version to 2.x helps for some reason. Checkout the demo for easy testing of the issue from the original issue

git clone -b charmcraft-bionic-bug https://git.launchpad.net/~esunar/charm-graylog
make charm3 #fails
make charm2 #sucess

Also reproduces for fstab-config charm. Probably other reactive charms are affected too.

We cannot pack the charm in 18.04(with charm 3.x) because charm snap needs python>3.7 and 18.04 has python 3.6.9
We cannot deploy it into bionic if we build with charm snap 3.x build on 22.04 because this wheelhouse is built from the basic layer and built on 22.04 that has python > 3.7

The correct way of dealing with this is to specify charm bases in the charmcraft.yaml (https://discourse.charmhub.io/t/charmcraft-bases-provider-support). If you specify 18.04 as your base you will get the expected result.

Previously the python in the snap was used when populating the charm wheelhouse, and incidentally the snap used a core18 base, which is why this used to work for you.

Tying a charm's wheelhouse to the base used in the charm-tools snap is not maintainable so the default has been changed to use the Python in the build environment. As noted in the changelog you can use the --use-python-from-snap argument to restore the old behavior, but given the charm snap now uses a core22 base (Python 3.10) it would not resolve your issue.

juju / charm-tools

Reactive charms built with charm version 3.x does not install on bionic (Python >=3.7 is required, 3.6.9 is present) #640