Closed nate-byrnes closed 7 years ago
Hi,
The introduce
function has been added with update 7826d00
Also you can introduce all forgotten VDIs using xe sr-scan
An explicit introduce yielded:
Jun 7 17:56:36 xen3 SM: [8790] vdi_introduce {'sr_uuid': 'ab0fe076-12f7-4a7e-8792-73a70a7a2301', 'subtask_of': 'DummyRef:|7ad161c8-a3ce-2107-8216-ba4adeb1ee43|VDI.introduce', 'args': [], 'vdi_location': '939a5426-51aa-42f2-91bd-13ce4e1fa2e5', 'host_ref': 'OpaqueRef:f9942fa1-4705-0914-0468-
fb54bcbdaa68', 'session_ref': 'OpaqueRef:93734112-00a9-b39f-4029-de72c5905f98', 'device_config': {'SRmaster': 'true'}, 'command': 'vdi_introduce', 'sr_ref': 'OpaqueRef:ddd35779-b212-ecf2-486d-83a0be957f9c', 'new_uuid': '939a5426-51aa-42f2-91bd-13ce4e1fa2e5', 'vdi_sm_config': {'vdi_type': 'a
io'}}
Jun 7 17:56:36 xen3 SM: [8790] RBDVDI.load: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun 7 17:56:36 xen3 SM: [8790] Calling cephutils.SR._get_path: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun 7 17:56:36 xen3 SM: [8790] Calling cephutils.VDI.load: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun 7 17:56:36 xen3 SM: [8790] RBDVDI.introduce: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun 7 17:56:36 xen3 SM: [8790] ***** generic exception: vdi_introduce: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun 7 17:56:36 xen3 SM: [8790] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 7 17:56:36 xen3 SM: [8790] return self._run_locked(sr)
Jun 7 17:56:36 xen3 SM: [8790] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 7 17:56:36 xen3 SM: [8790] rv = self._run(sr, target)
Jun 7 17:56:36 xen3 SM: [8790] File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun 7 17:56:36 xen3 SM: [8790] return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun 7 17:56:36 xen3 SM: [8790] File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun 7 17:56:36 xen3 SM: [8790] if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
Jun 7 17:56:36 xen3 SM: [8790]
And sr-scan ended up with:
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-de012844-3557-4c18-962b-026bc8887350
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e2d4795e-35c3-4790-90ab-a21ec1b4e30e
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e60697ef-3d9b-4d81-9be1-cb0af9bfcb85
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e7888911-bf36-4282-8fe7-9a92b2de5074
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-eac39c20-7dbb-4747-b6ec-3fb4dfe83b1f
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-ed9ce8e5-4a75-416c-8d4e-a7eaf8749905
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-edf530c2-4396-4ced-93af-2a1939f93ea4
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-f2af44da-a5e7-4893-bb4c-28be23e3e275
Jun 7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-f8def2ff-3cd4-45e2-9df9-c36881f305e9
Jun 7 18:02:57 xen3 SM: [15709] ***** generic exception: sr_scan: EXCEPTION <type 'exceptions.AttributeError'>, 'RBDSR' object has no attribute '_get_vdi_meta'
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 7 18:02:57 xen3 SM: [15709] return self._run_locked(sr)
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 7 18:02:57 xen3 SM: [15709] rv = self._run(sr, target)
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 339, in _run
Jun 7 18:02:57 xen3 SM: [15709] return sr.scan(self.params['sr_uuid'])
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/RBDSR", line 240, in scan
Jun 7 18:02:57 xen3 SM: [15709] self._loadvdis()
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/RBDSR", line 153, in _loadvdis
Jun 7 18:02:57 xen3 SM: [15709] vdi_meta = self._get_vdi_meta(vdi_uuid)
Jun 7 18:02:57 xen3 SM: [15709]
Jun 7 18:02:57 xen3 SM: [15709] ***** RBD: EXCEPTION <type 'exceptions.AttributeError'>, 'RBDSR' object has no attribute '_get_vdi_meta'
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 353, in run
Jun 7 18:02:57 xen3 SM: [15709] ret = cmd.run(sr)
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 7 18:02:57 xen3 SM: [15709] return self._run_locked(sr)
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 7 18:02:57 xen3 SM: [15709] rv = self._run(sr, target)
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/SRCommand.py", line 339, in _run
Jun 7 18:02:57 xen3 SM: [15709] return sr.scan(self.params['sr_uuid'])
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/RBDSR", line 240, in scan
Jun 7 18:02:57 xen3 SM: [15709] self._loadvdis()
Jun 7 18:02:57 xen3 SM: [15709] File "/opt/xensource/sm/RBDSR", line 153, in _loadvdis
Jun 7 18:02:57 xen3 SM: [15709] vdi_meta = self._get_vdi_meta(vdi_uuid)
Jun 7 18:02:57 xen3 SM: [15709]
Could I have not completed the installation correctly? I ran install.sh install jewel
and then xe-toolstack-restart
on all nodes. Do I need to reboot?
It looks like the _get_vdi_meta function is only defined in the VDI class, but it is requested as part of the SR class. Does SR inherit VDI someplace???
this seems to have gotten me past that issue:
diff -p /gluster/addons/RBDSR_current/bins/cephutils.py cephutils.py
*** /gluster/addons/RBDSR_current/bins/cephutils.py 2017-06-07 15:47:21.000000000 -0400
--- cephutils.py 2017-06-07 18:24:10.372056090 -0400
*************** class SR:
*** 75,80 ****
--- 75,93 ----
else:
return {}
+ def _get_vdi_meta(self, vdi_uuid):
+ util.SMlog("Calling cephutils.SR._get_vdi_meta: vdi_uuid=%s" % vdi_uuid)
+ vdi_name = "%s%s" % (VDI_PREFIX, vdi_uuid)
+ if self.use_rbd_meta:
+ cmdout = util.pread2(["rbd", "image-meta", "list", vdi_name, "--pool", self.CEPH_POOL_NAME, "--format", "json", "--name", self.CEPH_USER])
+ if len(cmdout) != 0:
+ decoded = json.loads(cmdout)
+ return decoded
+ else:
+ return {}
+ else:
+ return {}
+
def _get_vdilist(self, pool):
util.SMlog("Calling cephutils.SR._get_vdilist: pool=%s" % pool)
RBDVDIs = {}
introduce is giving me this error:
Jun 7 19:08:19 xen3 SM: [27906] RBDVDI.introduce: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=0420ea5b-ece0-4059-914b-4e780a0b5bf8
Jun 7 19:08:19 xen3 SM: [27906] ***** generic exception: vdi_introduce: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 7 19:08:19 xen3 SM: [27906] return self._run_locked(sr)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 7 19:08:19 xen3 SM: [27906] rv = self._run(sr, target)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun 7 19:08:19 xen3 SM: [27906] return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun 7 19:08:19 xen3 SM: [27906] if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
Jun 7 19:08:19 xen3 SM: [27906]
Jun 7 19:08:19 xen3 SM: [27906] ***** RBD: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 353, in run
Jun 7 19:08:19 xen3 SM: [27906] ret = cmd.run(sr)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 7 19:08:19 xen3 SM: [27906] return self._run_locked(sr)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 7 19:08:19 xen3 SM: [27906] rv = self._run(sr, target)
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun 7 19:08:19 xen3 SM: [27906] return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun 7 19:08:19 xen3 SM: [27906] File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun 7 19:08:19 xen3 SM: [27906] if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
Hi, Sorry, there were some typos that I didn't check.
I completely understand. You'll hear no complaints from me. I'm guessing that some of my meta data is messed up through my experimentation... do you have any ideas how to address the following? (Or, should I open another issue?)
Jun 8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-f60dd3ac-50e9-4a27-8465-51374131de5d
Jun 8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-ab0fe076-12f7-4a7e-8792-73a70a7a2301
Jun 8 08:10:40 xen5 SM: [8958] RBDVDI.load: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_path: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:40 xen5 SM: [8958] Calling cephutils.VDI.load: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:40 xen5 SM: [8958] RBDVDI.attach: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_path: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:41 xen5 SM: [8958] Calling cephutills.VDI._map_SNAP: vdi_uuid=b68eeac1-b6b7-40c5-b232-4157ef54f437, snap_uuid=c87fe31a-3a00-409b-af05-208c08904d8b, size=8589934592, dm=none, sharable=false
Jun 8 08:10:41 xen5 SM: [8958] Calling cephutils.VDI._call_plugin: op=map
Jun 8 08:10:41 xen5 SM: [8958] Calling ceph_plugin
Jun 8 08:10:41 xen5 SM: [8958] Calling 'map' on host OpaqueRef:7674197b-f969-54ca-8415-44a583252d5c
Jun 8 08:10:41 xen5 SM: [8958] Exception in activate/attach
Jun 8 08:10:41 xen5 SM: [8958] Removed host key host_OpaqueRef:d4828dc4-fcb9-c6ab-fcc0-94113d822fd2 for c87fe31a-3a00-409b-af05-208c08904d8b
Jun 8 08:10:41 xen5 SM: [8958] ***** BLKTAP2:<function _activate_locked at 0x112d6e0>: EXCEPTION <class 'XenAPI.Failure'>, ['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 87, in wrapper
Jun 8 08:10:41 xen5 SM: [8958] ret = op(self, *args)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 1574, in _activate_locked
Jun 8 08:10:41 xen5 SM: [8958] self._attach(sr_uuid, vdi_uuid)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 1639, in _attach
Jun 8 08:10:41 xen5 SM: [8958] attach_info = xmlrpclib.loads(self.target.attach(sr_uuid, vdi_uuid))[0][0]
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 1112, in attach
Jun 8 08:10:41 xen5 SM: [8958] return self.vdi.attach(sr_uuid, vdi_uuid)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/RBDSR", line 437, in attach
Jun 8 08:10:41 xen5 SM: [8958] self._map_SNAP(base_uuid, vdi_uuid, self.size, "none")
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/cephutils.py", line 618, in _map_SNAP
Jun 8 08:10:41 xen5 SM: [8958] self.session.xenapi.VDI.add_to_sm_config(snap_ref, 'dm', dm)
Jun 8 08:10:41 xen5 SM: [8958] File "/usr/lib/python2.7/site-packages/XenAPI.py", line 254, in __call__
Jun 8 08:10:41 xen5 SM: [8958] return self.__send(self.__name, args)
Jun 8 08:10:41 xen5 SM: [8958] File "/usr/lib/python2.7/site-packages/XenAPI.py", line 150, in xenapi_request
Jun 8 08:10:41 xen5 SM: [8958] result = _parse_result(getattr(self, methodname)(*full_params))
Jun 8 08:10:41 xen5 SM: [8958] File "/usr/lib/python2.7/site-packages/XenAPI.py", line 228, in _parse_result
Jun 8 08:10:41 xen5 SM: [8958] raise Failure(result['ErrorDescription'])
Jun 8 08:10:41 xen5 SM: [8958]
Jun 8 08:10:41 xen5 SM: [8958] Raising exception [46, The VDI is not available [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']]]
Jun 8 08:10:41 xen5 SM: [8958] lock: released /var/lock/sm/c87fe31a-3a00-409b-af05-208c08904d8b/vdi
Jun 8 08:10:41 xen5 SM: [8958] ***** generic exception: vdi_activate: EXCEPTION <class 'SR.SROSError'>, The VDI is not available [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']]
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun 8 08:10:41 xen5 SM: [8958] return self._run_locked(sr)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun 8 08:10:41 xen5 SM: [8958] rv = self._run(sr, target)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/SRCommand.py", line 264, in _run
Jun 8 08:10:41 xen5 SM: [8958] writable, caching_params)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 1541, in activate
Jun 8 08:10:41 xen5 SM: [8958] if self._activate_locked(sr_uuid, vdi_uuid, options):
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/blktap2.py", line 95, in wrapper
Jun 8 08:10:41 xen5 SM: [8958] raise xs_errors.XenError(excType, opterr=msg)
Jun 8 08:10:41 xen5 SM: [8958] File "/opt/xensource/sm/xs_errors.py", line 52, in __init__
Jun 8 08:10:41 xen5 SM: [8958] raise SR.SROSError(errorcode, errormessage)
Jun 8 08:10:41 xen5 SM: [8958]
Judging by the error RBDSR tries to add dm
key to sm_config but
sm_config` have such key already.
So you used right approach. If you forget and reintroduce the vdi it should have clean metadata after that.
Did the "forget/introduce" approach help you to resolve the problem?
That did it. (Forget, introduce that is).
It seems to me that whenever I force shutdown a guest, I need to perform this process of forget then introduce.... I'm still focused on some production repairs, so I haven't been able to thoroughly test this yet, so I thought I'd mention it in case it was something you were familiar with.
It seems pretty consistent that a force shutdown requires me to forget then introduce the VDI's of the machine shutdown. I think that the sm_config is not being re-set for whatever reason under force shutdown circumstances. That said, forget/introduce is now working, so, I'll close this issue.
Hello, while I am in the process of trying to re-attach a somehow lost VDI connection, I tested creating a simple VDI, forgetting it, and then introducing it again. (This was done in the hopes of being a way to re-connect my VDI that now will no longer boot....)
I can see that the RBD exists in my ceph cluster
When I run the introduce command:
The introduction fails, and SMlog has the following: