juju / amulet

Testing harness and tools for Juju Charms
Other
17 stars 33 forks source link

Surface Amulet failures differently than test failures #163

Closed johnsca closed 8 years ago

johnsca commented 8 years ago

We had a test run where Amulet failed due to SSH to a unit to upload filesystem_data.py failing. This is currently considered a test failure, but is really an infrastructure failure and should be reported as such. Amulet should handle failures like these and skip the test instead of failing.

2016-11-04 00:39:46 Deploying applications...
2016-11-04 00:40:53 Config specifies num units for subordinate: ganglia-node
2016-11-04 00:41:58 Config specifies num units for subordinate: plugin
2016-11-04 00:43:03 Config specifies num units for subordinate: rsyslog-forwarder-ha
2016-11-04 00:43:07 Adding relations...
2016-11-04 00:43:08 Deployment complete in 203.54 seconds
ERROR exit status 1 (ssh: connect to host 130.211.49.15 port 22: Connection timed out
lost connection)
/var/lib/jenkins/.local/lib/python3.5/site-packages/path.py:1717: DeprecationWarning: path is deprecated. Use Path instead.
  warnings.warn(msg, DeprecationWarning)
E
======================================================================
ERROR: setUpClass (__main__.TestBundle)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tmp.ibzsla4Olj/bundletester-dSTKSe/tmpqdqpHz/tests/01-bundle.py", line 49, in setUpClass
    cls.d.setup(timeout=3600)
  File "/var/lib/jenkins/.local/lib/python3.5/site-packages/amulet/deployer.py", line 677, in setup
    self.services, timeout=timeout, juju_env=self.juju_env)
  File "/var/lib/jenkins/.local/lib/python3.5/site-packages/amulet/sentry.py", line 359, in __init__
    self.unit[unit] = UnitSentry.fromunitdata(unit, unit_data)
  File "/var/lib/jenkins/.local/lib/python3.5/site-packages/amulet/sentry.py", line 82, in fromunitdata
    unitsentry.upload_scripts()
  File "/var/lib/jenkins/.local/lib/python3.5/site-packages/amulet/sentry.py", line 121, in upload_scripts
    subprocess.check_call(cmd.split())
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['juju', 'scp', '/var/lib/jenkins/.local/lib/python3.5/site-packages/amulet/unit-scripts/amulet/filesystem_data.py', 'flume-kafka/0:/tmp/amulet']' returned non-zero exit status 1

----------------------------------------------------------------------
Ran 0 tests in 886.875s

FAILED (errors=1)