search

openstack-charmers / zaza-openstack-tests

OpenStack Charms Functional Test Library for Zaza
Apache License 2.0
7 stars 77 forks source link

[ubuntu-lite] series upgrade (bionic) "do-release-upgrade" failed #358

Open ajkavanagh opened 4 years ago

ajkavanagh commented 4 years ago

Unfortunately, no artifacts from jenkins, but the trace report is:

...
03:40:12 func runtests: commands[0] | functest-run-suite --keep-model
03:40:16 2020-07-06 03:40:16 [INFO] Deploying bundle './tests/bundles/bundle.yaml' on to 'zaza-c610b240ee52' model
03:40:17 2020-07-06 03:40:17 [INFO] Resolving charm: cs:~chris.macnaughton/test-ubuntu
03:40:18 2020-07-06 03:40:18 [INFO] Executing changes:
03:40:18 2020-07-06 03:40:18 [INFO] - upload charm cs:~chris.macnaughton/test-ubuntu-1 for series bionic
03:40:18 2020-07-06 03:40:18 [INFO] - deploy application ubuntu-lite on bionic using cs:~chris.macnaughton/test-ubuntu-1
03:40:19 2020-07-06 03:40:19 [INFO] - add unit ubuntu-lite/0 to new machine 0
03:40:21 2020-07-06 03:40:21 [INFO] - add unit ubuntu-lite/1 to new machine 1
03:40:22 2020-07-06 03:40:22 [INFO] - add unit ubuntu-lite/2 to new machine 2

...

03:42:32 2020-07-06 03:42:32 [INFO] Upgrading 2
03:42:32 2020-07-06 03:42:32 [INFO] About to call '['juju', 'run', '--machine=2', '--timeout=120m', 'yes | sudo DEBIAN_FRONTEND=noninteractive do-release-upgrade -f DistUpgradeViewNonInteractive']'
03:42:32 2020-07-06 03:42:32 [INFO] STDOUT: Reading package lists...
03:42:32 Building dependency tree...
03:42:32 Reading state information...
03:42:32 Calculating upgrade...

...

03:42:34 2020-07-06 03:42:34 [INFO] ======================================================================
03:42:34 2020-07-06 03:42:34 [INFO] ERROR: test_200_run_series_upgrade (zaza.openstack.charm_tests.series_upgrade.parallel_tests.BionicFocalSeriesUpgradeUbuntu)
03:42:34 2020-07-06 03:42:34 [INFO] Run series upgrade.
03:42:34 2020-07-06 03:42:34 [INFO] ----------------------------------------------------------------------
03:42:34 2020-07-06 03:42:34 [INFO] Traceback (most recent call last):
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/openstack/charm_tests/series_upgrade/parallel_tests.py", line 187, in test_200_run_series_upgrade
03:42:34 2020-07-06 03:42:34 [INFO]     to_series=self.to_series
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/openstack/utilities/parallel_series_upgrade.py", line 124, in upgrade_ubuntu_lite
03:42:34 2020-07-06 03:42:34 [INFO]     completed_machines=completed_machines, origin=None)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
03:42:34 2020-07-06 03:42:34 [INFO]     return future.result()
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
03:42:34 2020-07-06 03:42:34 [INFO]     raise self._exception
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
03:42:34 2020-07-06 03:42:34 [INFO]     result = coro.throw(exc)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/openstack/utilities/parallel_series_upgrade.py", line 231, in parallel_series_upgrade
03:42:34 2020-07-06 03:42:34 [INFO]     await asyncio.gather(*upgrade_group)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/futures.py", line 361, in __iter__
03:42:34 2020-07-06 03:42:34 [INFO]     yield self  # This tells Task to wait for completion.
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/tasks.py", line 296, in _wakeup
03:42:34 2020-07-06 03:42:34 [INFO]     future.result()
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/futures.py", line 274, in result
03:42:34 2020-07-06 03:42:34 [INFO]     raise self._exception
03:42:34 2020-07-06 03:42:34 [INFO]   File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
03:42:34 2020-07-06 03:42:34 [INFO]     result = coro.send(None)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/openstack/utilities/parallel_series_upgrade.py", line 389, in series_upgrade_machine
03:42:34 2020-07-06 03:42:34 [INFO]     await async_do_release_upgrade(machine)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/openstack/utilities/parallel_series_upgrade.py", line 606, in async_do_release_upgrade
03:42:34 2020-07-06 03:42:34 [INFO]     machine, do_release_upgrade_cmd, timeout="120m")
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/model.py", line 1918, in async_run_on_machine
03:42:34 2020-07-06 03:42:34 [INFO]     await generic_utils.check_call(cmd)
03:42:34 2020-07-06 03:42:34 [INFO]   File "/tmp/tmp.xo1A87ktvc/func/lib/python3.5/site-packages/zaza/utilities/generic.py", line 678, in check_call
03:42:34 2020-07-06 03:42:34 [INFO]     raise subprocess.CalledProcessError(proc.returncode, cmd)
03:42:34 2020-07-06 03:42:34 [INFO] subprocess.CalledProcessError: Command '['juju', 'run', '--machine=0', '--timeout=120m', 'yes | sudo DEBIAN_FRONTEND=noninteractive do-release-upgrade -f DistUpgradeViewNonInteractive']' returned non-zero exit status 1

Note there is no technical difference between the machines; it just looks like the 3rd one failed. Raising bug as a way to track if this happens more frequently.

andrewdmcleod commented 4 years ago

ive hit this again