termie / nova-migration-demo

Nova is a cloud computing fabric controller (the main part of an IaaS system). It is written in Python.
http://openstack.org/projects/compute/
Apache License 2.0
2 stars 0 forks source link

xenstore.py inadequate exception handling #845

Open termie opened 13 years ago

termie commented 13 years ago

Revision 989 (Cactus).

I have got an exception during the launch of an instance. I get the following stacktrace in nova-compute.log:

2011-04-21 22:47:12,887 DEBUG nova [-] injecting network info to xs for vm: |OpaqueRef:31c1cdee-b215-470a-8a93-07599fb4ab28| from (pid=2349) _inject_network_info /usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py:859 2011-04-21 22:47:13,606 WARNING nova.virt.xenapi [-] Task [Async.host.call_plugin] OpaqueRef:6897e357-a275-828b-fe22-74ffe95df056 status: failure ['XENAPI_PLUGIN_FAILURE', 'non-zero exit', '', ''] 2011-04-21 22:47:13,612 ERROR nova.compute.manager [58X1QE5AKBX4UWIBNJRX dashboard dashboard] Instance '1' failed to spawn. Is virtualization enabled in the BIOS? (nova.compute.manager): TRACE: Traceback (most recent call last): (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 234, in run_instance (nova.compute.manager): TRACE: self.driver.spawn(instance_ref) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi_conn.py", line 188, in spawn (nova.compute.manager): TRACE: self._vmops.spawn(instance) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py", line 118, in spawn (nova.compute.manager): TRACE: vm_ref = self._create_vm(instance, vdi_uuid, network_info) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py", line 179, in _create_vm (nova.compute.manager): TRACE: self.inject_network_info(instance, network_info, vm_ref) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py", line 850, in inject_network_info (nova.compute.manager): TRACE: self._inject_network_info(instance, network_info, vm_ref) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py", line 875, in _inject_network_info (nova.compute.manager): TRACE: vm_ref) (nova.compute.manager): TRACE: File "/usr/lib/python2.6/site-packages/nova/virt/xenapi/vmops.py", line 977, in _make_plugin_call (nova.compute.manager): TRACE: err_msg = err_trace.splitlines()-1: TRACE: IndexError: list index out of range (nova.compute.manager): TRACE:

Two points here:

1) "Instance '1' failed to spawn. Is virtualization enabled in the BIOS?" as error message is misleading and does not really a clue of what's going on 2) There's an unhandled exception within the catch block in vmops.py:977

As for /var/log/messages on the host, I get:

Apr 22 03:00:39 localhost fe: 20110422T03:00:39.664Z||5861|Args after replacement = [/etc/xapi.d/plugins/xenstore.py;write_recordOpaqueRef:8985f82c-2c1a-bf2f-d452-8d19b2589b3epathvm-data/networking/02163e0e00b9dom_id-1value{"label": "public", "broadcast": "10.0.0.31", "ips": [{"ip": "10.0.0.2", "netmask": "255.255.255.224", "enabled": "1"}], "mac": "02:16:3e:0e:00:b9", "dns": ["192.168.1.1"], "gateway": "10.0.0.1"}] Apr 22 03:00:39 localhost fe: 20110422T03:00:39.664Z||5861|I've received the following fds: [2;1] Apr 22 03:00:39 localhost fe: 20110422T03:00:39.666Z||5862|Caught unexpected exception: Unix.Unix_error(1, "execve", "/etc/xapi.d/plugins/xenstore.py") Apr 22 03:00:39 localhost fe: 5862 (/etc/xapi.d/plugins/xenstore.py write_record</methodN...) exitted with code 1

No exception handling, that let us get a clue of what's going on.

Any idea of what might have gone wrong? Thanks, Armando