Collecting logs is not completely safe. When the connection is interrupted the whole log collecting will raise causing a complete restart.
We should capture these errors, log them and then proceed testing
Error found:
"2018-02-07 03:11:15.733260", "failed": true, "rc": 1, "start": "2018-02-06 23:40:10.012215", "stderr": "libvirt: QEMU Driver error : Domain not found: no domain with matching name 'HA-test-000'
libvirt: QEMU Driver error : Domain not found: no domain with matching name 'mds-regression-000'
Traceback (most recent call last):
File \"<string>\", line 1, in <module>
File \"/opt/OpenvStorage/ci/main.py\", line 328, in main
w.run()
File \"/opt/OpenvStorage/ci/main.py\", line 82, in run
Workflow._run_with_retry(self.scenario, self.config['ci']['scenario_retries'])
File \"/opt/OpenvStorage/ci/main.py\", line 69, in _run_with_retry
target(*args, **kwargs)
File \"/opt/OpenvStorage/ci/main.py\", line 270, in scenario
exclude_scenarios=exclude_scenarios)
File \"/opt/OpenvStorage/ci/autotests.py\", line 73, in run
module_result = mod.run(blocked)
File \"/opt/OpenvStorage/ci/scenarios/arakoon/ovs_3671_validate_archiving_of_existing_arakoon_data_on_create_and_extend_test/main.py\", line 155, in run
return ArakoonArchiving().main(blocked)
File \"/opt/OpenvStorage/ci/autotests.py\", line 459, in wrapped
result = [str(ex), '', 'Logs collected between {0} and {1}'.format(start, end), '', LogCollector.get_logs(components=log_components, since=start, until=end)]
File \"/opt/OpenvStorage/ci/autotests.py\", line 407, in get_logs
return LogFileTimeParser.execute_search_on_remote(since=since, until=until, search_locations=units)
File \"/usr/lib/python2.7/dist-packages/ovs_extensions/log/log_reader.py\", line 139, in execute_search_on_remote
with remote(hosts, [LogFileTimeParser], username=username, password=password) as remotes:
File \"/usr/lib/python2.7/dist-packages/ovs_extensions/generic/remote.py\", line 66, in __init__
self.machines = [SshMachine(ip, user=self.username, password=password, ssh_opts=tuple(ssh_opts)) for ip in self.ips]
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/ssh_machine.py\", line 106, in __init__
new_session = new_session)
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/remote.py\", line 129, in __init__
self._session = self.session(new_session = new_session)
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/ssh_machine.py\", line 150, in session
self.encoding, isatty, self.connect_timeout)
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/session.py\", line 141, in __init__
self.run(\"\")
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/session.py\", line 224, in run
return run_proc(self.popen(cmd), retcode)
File \"/usr/lib/python2.7/dist-packages/plumbum/commands/processes.py\", line 186, in run_proc
stdout, stderr = proc.communicate()
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/session.py\", line 92, in communicate
line = pipe.readline()
File \"/usr/lib/python2.7/dist-packages/plumbum/machines/session.py\", line 42, in readline
raise EOFError()
EOFError"
Problem description
Collecting logs is not completely safe. When the connection is interrupted the whole log collecting will raise causing a complete restart. We should capture these errors, log them and then proceed testing
Error found: