Reactive charms cannot handle a series upgrade

[ChrisMacNaughton]

When upgrading from Trusty to Xenial, the ubuntu charm fails with:

2018-03-14 14:12:07 DEBUG update-status Traceback (most recent call last):
2018-03-14 14:12:07 DEBUG update-status   File "/var/lib/juju/agents/unit-ubuntu-0/charm/hooks/update-status", line 9, in <module>
2018-03-14 14:12:07 DEBUG update-status     basic.init_config_states()
2018-03-14 14:12:07 DEBUG update-status   File "lib/charms/layer/basic.py", line 174, in init_config_states
2018-03-14 14:12:07 DEBUG update-status     from charmhelpers.core import hookenv
2018-03-14 14:12:07 DEBUG update-status ImportError: No module named 'charmhelpers'
2018-03-14 14:12:07 ERROR juju.worker.uniter.operation runhook.go:113 hook "update-status" failed: exit status 1

[ChrisMacNaughton]

Perhaps the file checked at https://github.com/juju-solutions/layer-basic/blob/master/lib/charms/layer/basic.py#L38 could additionally include either python or Ubuntu versions, so that we can check if the world changes underneath us

[johnsca]

This is likely mitigated by #105, since the venv will still exist and effectively pins the Python version and libraries, so it would be worth testing with a rebuilt ubuntu charm. However, the use_venv: false case is still broken.

[ChrisMacNaughton]

Apparently, even when using a virtualenv, they aren't too happy:

2018-03-30 12:41:25 DEBUG update-status Could not find platform independent libraries <prefix>
2018-03-30 12:41:25 DEBUG update-status Could not find platform dependent libraries <exec_prefix>
2018-03-30 12:41:25 DEBUG update-status Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
2018-03-30 12:41:25 DEBUG update-status Fatal Python error: Py_Initialize: Unable to get the locale encoding
2018-03-30 12:41:25 DEBUG update-status ImportError: No module named 'encodings'
2018-03-30 12:41:25 ERROR juju.worker.uniter.operation runhook.go:113 hook "update-status" failed: signal: aborted

[ChrisMacNaughton]

The Juju docs have been updated to include running sudo rm /var/lib/juju/agents/unit*/charm/wheelhouse/.bootstrapped as part of the upgrade; however, doing that doesn't seem to resolve the above linked trace

[johnsca]

@ChrisMacNaughton What about also doing sudo rm -rf /var/lib/juju/agents/unit*/.venv?

[ChrisMacNaughton]

@johnsca still hits the same error after removing that specified wildcard

[ChrisMacNaughton]

@johnsca I managed to get a new error by adding a juju resolved to the calls:

2018-05-03 15:33:58 DEBUG update-status Traceback (most recent call last):
2018-05-03 15:33:58 DEBUG update-status   File "/var/lib/juju/agents/unit-magpie-2/charm/hooks/update-status", line 8, in <module>
2018-05-03 15:33:58 DEBUG update-status     basic.bootstrap_charm_deps()
2018-05-03 15:33:58 DEBUG update-status   File "lib/charms/layer/basic.py", line 84, in bootstrap_charm_deps
2018-05-03 15:33:58 DEBUG update-status     'pip'])
2018-05-03 15:33:58 DEBUG update-status   File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
2018-05-03 15:33:58 DEBUG update-status     raise CalledProcessError(retcode, cmd)
2018-05-03 15:33:58 DEBUG update-status subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-magpie-2/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status -6

[ChrisMacNaughton]

@johnsca after doing a juju resolved, it throws again on the next hook execution:

2018-05-04 15:06:43 DEBUG update-status Reading package lists...
2018-05-04 15:06:43 DEBUG update-status Building dependency tree...
2018-05-04 15:06:43 DEBUG update-status Reading state information...
2018-05-04 15:06:43 DEBUG update-status python3-dev is already the newest version (3.5.1-3).
2018-05-04 15:06:43 DEBUG update-status python3-setuptools is already the newest version (20.7.0-1).
2018-05-04 15:06:43 DEBUG update-status python3-yaml is already the newest version (3.11-3build1).
2018-05-04 15:06:43 DEBUG update-status python3-pip is already the newest version (8.1.1-2ubuntu0.4).
2018-05-04 15:06:43 DEBUG update-status 0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
2018-05-04 15:06:43 DEBUG update-status Reading package lists...
2018-05-04 15:06:44 DEBUG update-status Building dependency tree...
2018-05-04 15:06:44 DEBUG update-status Reading state information...
2018-05-04 15:06:44 DEBUG update-status 0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
2018-05-04 15:06:44 DEBUG update-status Could not find platform independent libraries <prefix>
2018-05-04 15:06:44 DEBUG update-status Could not find platform dependent libraries <exec_prefix>
2018-05-04 15:06:44 DEBUG update-status Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
2018-05-04 15:06:44 DEBUG update-status Fatal Python error: Py_Initialize: Unable to get the locale encoding
2018-05-04 15:06:44 DEBUG update-status ImportError: No module named 'encodings'
2018-05-04 15:06:44 DEBUG update-status Traceback (most recent call last):
2018-05-04 15:06:44 DEBUG update-status   File "/var/lib/juju/agents/unit-magpie-0/charm/hooks/update-status", line 8, in <module>
2018-05-04 15:06:44 DEBUG update-status     basic.bootstrap_charm_deps()
2018-05-04 15:06:44 DEBUG update-status   File "lib/charms/layer/basic.py", line 84, in bootstrap_charm_deps
2018-05-04 15:06:44 DEBUG update-status     'pip'])
2018-05-04 15:06:44 DEBUG update-status   File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
2018-05-04 15:06:44 DEBUG update-status     raise CalledProcessError(retcode, cmd)
2018-05-04 15:06:44 DEBUG update-status subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-magpie-0/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'pip']' returned non-zero exit status -6

[afreiberger]

I found removing the .bootstrapped file from wheelhouse and then restarting the jujud unit affected did resolve this for me.

[johnsca]

This was fixed in #115 but some improvements could be made, per #158