search

rposudnevskiy / RBDSR

RBDSR - XenServer/XCP-ng Storage Manager plugin for CEPH
GNU Lesser General Public License v2.1
58 stars 23 forks source link

Unable to introduce a forgotten VDI #49

Closed nate-byrnes closed 7 years ago

nate-byrnes commented 7 years ago

Hello, while I am in the process of trying to re-attach a somehow lost VDI connection, I tested creating a simple VDI, forgetting it, and then introducing it again. (This was done in the hopes of being a way to re-connect my VDI that now will no longer boot....)

I can see that the RBD exists in my ceph cluster

[root@xen5 ~]# rbd image-meta list VHD-551f9479-940b-4d95-a0d2-c1beb3f53a08 --pool RBD_XenStorage-ab0fe076-12f7-4a7e-8792-73a70a7a2301                                             
There are 2 metadatas on this image.
Key             Value                                                             
VDI_DESCRIPTION Test disk image for the purposes of forgetting and re-introducing 
VDI_LABEL       NB_TEST_FORGET_INTRO                                              
[root@xen5 ~]#

When I run the introduce command: 

 xe vdi-introduce name-label=NB_TESTED_NOW_INTROD name-description="Test disk image forgotten, now remembered" location=551f9479-940b-4d95-a0d2-c1beb3f53a08 type=user sr-uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301 uuid=551f9479-940b-4d95-a0d2-c1beb3f53a08 sm-config:vdi_type=aio 

# I get:

The server failed to handle your request, due to an internal error.  The given message may give details useful for debugging the problem.
message: Storage_interface.Vdi_does_not_exist("551f9479-940b-4d95-a0d2-c1beb3f53a08")

The introduction fails, and SMlog has the following:

Jun  6 22:43:52 xen3 SM: [20527] RBDSR.load: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301
Jun  6 22:43:52 xen3 SM: [20527] Calling cephutils.SR.load: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, ceph_user=admin
Jun  6 22:43:52 xen3 SM: [20527] Calling cephutils.SR._get_srlist
Jun  6 22:43:52 xen3 SM: [20527] ['ceph', 'df', '--format', 'json', '--name', 'client.admin']
Jun  6 22:43:53 xen3 SM: [20527]   pread SUCCESS
Jun  6 22:43:53 xen3 SM: [20527] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-f60dd3ac-50e9-4a27-8465-51374131de5d
Jun  6 22:43:53 xen3 SM: [20527] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-ab0fe076-12f7-4a7e-8792-73a70a7a2301
Jun  6 22:43:53 xen3 SM: [20527] vdi_introduce {'sr_uuid': 'ab0fe076-12f7-4a7e-8792-73a70a7a2301', 'subtask_of': 'DummyRef:|749726ce-a2fa-cb4e-6b50-bbfc75fa08bc|VDI.introduce', 'args': [], 'vdi_location': '551f9479-940b-4d95-a0d2-c1beb3f53a08', 'host_ref': 'OpaqueRef:f9942fa1-4705-0914-0468-fb54bcbdaa68', 'session_ref': 'OpaqueRef:743c4d30-3059-80b9-a5db-22a339b58eb5', 'device_config': {'SRmaster': 'true'}, 'command': 'vdi_introduce', 'sr_ref': 'OpaqueRef:ddd35779-b212-ecf2-486d-83a0be957f9c', 'new_uuid': '551f9479-940b-4d95-a0d2-c1beb3f53a08', 'vdi_sm_config': {'vdi_type': 'aio'}}
Jun  6 22:43:53 xen3 SM: [20527] RBDVDI.load: vdi_uuid=551f9479-940b-4d95-a0d2-c1beb3f53a08
Jun  6 22:43:53 xen3 SM: [20527] Calling cephutils.SR._get_path: vdi_uuid=551f9479-940b-4d95-a0d2-c1beb3f53a08
Jun  6 22:43:53 xen3 SM: [20527] Calling cephutils.VDI.load: vdi_uuid=551f9479-940b-4d95-a0d2-c1beb3f53a08
Jun  6 22:43:53 xen3 SM: [20527] Raising exception [38, The requested method is not supported/implemented]
Jun  6 22:43:53 xen3 SM: [20527] ***** generic exception: vdi_introduce: EXCEPTION <class 'SR.SROSError'>, The requested method is not supported/implemented
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  6 22:43:53 xen3 SM: [20527]     return self._run_locked(sr)
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  6 22:43:53 xen3 SM: [20527]     rv = self._run(sr, target)
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun  6 22:43:53 xen3 SM: [20527]     return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/VDI.py", line 163, in introduce
Jun  6 22:43:53 xen3 SM: [20527]     raise xs_errors.XenError('Unimplemented')
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/xs_errors.py", line 52, in __init__
Jun  6 22:43:53 xen3 SM: [20527]     raise SR.SROSError(errorcode, errormessage)
Jun  6 22:43:53 xen3 SM: [20527]
Jun  6 22:43:53 xen3 SM: [20527] ***** RBD: EXCEPTION <class 'SR.SROSError'>, The requested method is not supported/implemented
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 353, in run
Jun  6 22:43:53 xen3 SM: [20527]     ret = cmd.run(sr)
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  6 22:43:53 xen3 SM: [20527]     return self._run_locked(sr)
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  6 22:43:53 xen3 SM: [20527]     rv = self._run(sr, target)
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun  6 22:43:53 xen3 SM: [20527]     return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun  6 22:43:53 xen3 SM: [20527]   File "/opt/xensource/sm/VDI.py", line 163, in introduce
Jun  6 22:43:53 xen3 SM: [20527]     raise xs_errors.XenError('Unimplemented')
rposudnevskiy commented 7 years ago

Hi, The introduce function has been added with update 7826d00 Also you can introduce all forgotten VDIs using xe sr-scan

nate-byrnes commented 7 years ago

An explicit introduce yielded:

Jun  7 17:56:36 xen3 SM: [8790] vdi_introduce {'sr_uuid': 'ab0fe076-12f7-4a7e-8792-73a70a7a2301', 'subtask_of': 'DummyRef:|7ad161c8-a3ce-2107-8216-ba4adeb1ee43|VDI.introduce', 'args': [], 'vdi_location': '939a5426-51aa-42f2-91bd-13ce4e1fa2e5', 'host_ref': 'OpaqueRef:f9942fa1-4705-0914-0468-
fb54bcbdaa68', 'session_ref': 'OpaqueRef:93734112-00a9-b39f-4029-de72c5905f98', 'device_config': {'SRmaster': 'true'}, 'command': 'vdi_introduce', 'sr_ref': 'OpaqueRef:ddd35779-b212-ecf2-486d-83a0be957f9c', 'new_uuid': '939a5426-51aa-42f2-91bd-13ce4e1fa2e5', 'vdi_sm_config': {'vdi_type': 'a
io'}}
Jun  7 17:56:36 xen3 SM: [8790] RBDVDI.load: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun  7 17:56:36 xen3 SM: [8790] Calling cephutils.SR._get_path: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun  7 17:56:36 xen3 SM: [8790] Calling cephutils.VDI.load: vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun  7 17:56:36 xen3 SM: [8790] RBDVDI.introduce: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=939a5426-51aa-42f2-91bd-13ce4e1fa2e5
Jun  7 17:56:36 xen3 SM: [8790] ***** generic exception: vdi_introduce: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun  7 17:56:36 xen3 SM: [8790]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  7 17:56:36 xen3 SM: [8790]     return self._run_locked(sr)
Jun  7 17:56:36 xen3 SM: [8790]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  7 17:56:36 xen3 SM: [8790]     rv = self._run(sr, target)
Jun  7 17:56:36 xen3 SM: [8790]   File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun  7 17:56:36 xen3 SM: [8790]     return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun  7 17:56:36 xen3 SM: [8790]   File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun  7 17:56:36 xen3 SM: [8790]     if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
Jun  7 17:56:36 xen3 SM: [8790]
nate-byrnes commented 7 years ago

And sr-scan ended up with:

Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-de012844-3557-4c18-962b-026bc8887350
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e2d4795e-35c3-4790-90ab-a21ec1b4e30e
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e60697ef-3d9b-4d81-9be1-cb0af9bfcb85
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-e7888911-bf36-4282-8fe7-9a92b2de5074
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-eac39c20-7dbb-4747-b6ec-3fb4dfe83b1f
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-ed9ce8e5-4a75-416c-8d4e-a7eaf8749905
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-edf530c2-4396-4ced-93af-2a1939f93ea4
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-f2af44da-a5e7-4893-bb4c-28be23e3e275
Jun  7 18:02:57 xen3 SM: [15709] Calling cephutils.SR._get_vdi_uuid: vdi=VHD-f8def2ff-3cd4-45e2-9df9-c36881f305e9
Jun  7 18:02:57 xen3 SM: [15709] ***** generic exception: sr_scan: EXCEPTION <type 'exceptions.AttributeError'>, 'RBDSR' object has no attribute '_get_vdi_meta'
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  7 18:02:57 xen3 SM: [15709]     return self._run_locked(sr)
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  7 18:02:57 xen3 SM: [15709]     rv = self._run(sr, target)
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 339, in _run
Jun  7 18:02:57 xen3 SM: [15709]     return sr.scan(self.params['sr_uuid'])
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/RBDSR", line 240, in scan
Jun  7 18:02:57 xen3 SM: [15709]     self._loadvdis()
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/RBDSR", line 153, in _loadvdis
Jun  7 18:02:57 xen3 SM: [15709]     vdi_meta = self._get_vdi_meta(vdi_uuid)
Jun  7 18:02:57 xen3 SM: [15709]
Jun  7 18:02:57 xen3 SM: [15709] ***** RBD: EXCEPTION <type 'exceptions.AttributeError'>, 'RBDSR' object has no attribute '_get_vdi_meta'
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 353, in run
Jun  7 18:02:57 xen3 SM: [15709]     ret = cmd.run(sr)
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  7 18:02:57 xen3 SM: [15709]     return self._run_locked(sr)
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  7 18:02:57 xen3 SM: [15709]     rv = self._run(sr, target)
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/SRCommand.py", line 339, in _run
Jun  7 18:02:57 xen3 SM: [15709]     return sr.scan(self.params['sr_uuid'])
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/RBDSR", line 240, in scan
Jun  7 18:02:57 xen3 SM: [15709]     self._loadvdis()
Jun  7 18:02:57 xen3 SM: [15709]   File "/opt/xensource/sm/RBDSR", line 153, in _loadvdis
Jun  7 18:02:57 xen3 SM: [15709]     vdi_meta = self._get_vdi_meta(vdi_uuid)
Jun  7 18:02:57 xen3 SM: [15709]

Could I have not completed the installation correctly? I ran install.sh install jewel and then xe-toolstack-restart on all nodes. Do I need to reboot?

nate-byrnes commented 7 years ago

It looks like the _get_vdi_meta function is only defined in the VDI class, but it is requested as part of the SR class. Does SR inherit VDI someplace???

nate-byrnes commented 7 years ago

this seems to have gotten me past that issue:

diff -p /gluster/addons/RBDSR_current/bins/cephutils.py cephutils.py                                 
*** /gluster/addons/RBDSR_current/bins/cephutils.py     2017-06-07 15:47:21.000000000 -0400
--- cephutils.py        2017-06-07 18:24:10.372056090 -0400
*************** class SR:
*** 75,80 ****
--- 75,93 ----
          else:
               return {}

+     def _get_vdi_meta(self, vdi_uuid):
+         util.SMlog("Calling cephutils.SR._get_vdi_meta: vdi_uuid=%s" % vdi_uuid)
+         vdi_name = "%s%s" % (VDI_PREFIX, vdi_uuid)
+         if self.use_rbd_meta:
+             cmdout = util.pread2(["rbd", "image-meta", "list", vdi_name, "--pool", self.CEPH_POOL_NAME, "--format", "json", "--name", self.CEPH_USER])
+             if len(cmdout) != 0:
+                 decoded = json.loads(cmdout)
+                 return decoded
+             else:
+                 return {}
+         else:
+             return {}
+ 
      def _get_vdilist(self, pool):
          util.SMlog("Calling cephutils.SR._get_vdilist: pool=%s" % pool)
          RBDVDIs = {}
nate-byrnes commented 7 years ago

introduce is giving me this error:

Jun  7 19:08:19 xen3 SM: [27906] RBDVDI.introduce: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=0420ea5b-ece0-4059-914b-4e780a0b5bf8
Jun  7 19:08:19 xen3 SM: [27906] ***** generic exception: vdi_introduce: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  7 19:08:19 xen3 SM: [27906]     return self._run_locked(sr)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  7 19:08:19 xen3 SM: [27906]     rv = self._run(sr, target)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun  7 19:08:19 xen3 SM: [27906]     return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun  7 19:08:19 xen3 SM: [27906]     if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
Jun  7 19:08:19 xen3 SM: [27906]
Jun  7 19:08:19 xen3 SM: [27906] ***** RBD: EXCEPTION <type 'exceptions.TypeError'>, unbound method _if_vdi_exist() must be called with VDI instance as first argument (got str instance instead)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 353, in run
Jun  7 19:08:19 xen3 SM: [27906]     ret = cmd.run(sr)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  7 19:08:19 xen3 SM: [27906]     return self._run_locked(sr)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  7 19:08:19 xen3 SM: [27906]     rv = self._run(sr, target)
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/SRCommand.py", line 234, in _run
Jun  7 19:08:19 xen3 SM: [27906]     return target.introduce(self.params['sr_uuid'], self.params['new_uuid'])
Jun  7 19:08:19 xen3 SM: [27906]   File "/opt/xensource/sm/RBDSR", line 322, in introduce
Jun  7 19:08:19 xen3 SM: [27906]     if cephutils.VDI._if_vdi_exist(sr_uuid, vdi_uuid):
rposudnevskiy commented 7 years ago

Hi, Sorry, there were some typos that I didn't check.

nate-byrnes commented 7 years ago

I completely understand. You'll hear no complaints from me. I'm guessing that some of my meta data is messed up through my experimentation... do you have any ideas how to address the following? (Or, should I open another issue?)

Jun  8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-f60dd3ac-50e9-4a27-8465-51374131de5d
Jun  8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_sr_uuid_by_name: pool=RBD_XenStorage-ab0fe076-12f7-4a7e-8792-73a70a7a2301
Jun  8 08:10:40 xen5 SM: [8958] RBDVDI.load: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_path: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:40 xen5 SM: [8958] Calling cephutils.VDI.load: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:40 xen5 SM: [8958] RBDVDI.attach: sr_uuid=ab0fe076-12f7-4a7e-8792-73a70a7a2301, vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:40 xen5 SM: [8958] Calling cephutils.SR._get_path: vdi_uuid=c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:41 xen5 SM: [8958] Calling cephutills.VDI._map_SNAP: vdi_uuid=b68eeac1-b6b7-40c5-b232-4157ef54f437, snap_uuid=c87fe31a-3a00-409b-af05-208c08904d8b, size=8589934592, dm=none, sharable=false
Jun  8 08:10:41 xen5 SM: [8958] Calling cephutils.VDI._call_plugin: op=map
Jun  8 08:10:41 xen5 SM: [8958] Calling ceph_plugin
Jun  8 08:10:41 xen5 SM: [8958] Calling 'map' on host OpaqueRef:7674197b-f969-54ca-8415-44a583252d5c
Jun  8 08:10:41 xen5 SM: [8958] Exception in activate/attach
Jun  8 08:10:41 xen5 SM: [8958] Removed host key host_OpaqueRef:d4828dc4-fcb9-c6ab-fcc0-94113d822fd2 for c87fe31a-3a00-409b-af05-208c08904d8b
Jun  8 08:10:41 xen5 SM: [8958] ***** BLKTAP2:<function _activate_locked at 0x112d6e0>: EXCEPTION <class 'XenAPI.Failure'>, ['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 87, in wrapper
Jun  8 08:10:41 xen5 SM: [8958]     ret = op(self, *args)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 1574, in _activate_locked
Jun  8 08:10:41 xen5 SM: [8958]     self._attach(sr_uuid, vdi_uuid)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 1639, in _attach
Jun  8 08:10:41 xen5 SM: [8958]     attach_info = xmlrpclib.loads(self.target.attach(sr_uuid, vdi_uuid))[0][0]
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 1112, in attach
Jun  8 08:10:41 xen5 SM: [8958]     return self.vdi.attach(sr_uuid, vdi_uuid)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/RBDSR", line 437, in attach
Jun  8 08:10:41 xen5 SM: [8958]     self._map_SNAP(base_uuid, vdi_uuid, self.size, "none")
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/cephutils.py", line 618, in _map_SNAP
Jun  8 08:10:41 xen5 SM: [8958]     self.session.xenapi.VDI.add_to_sm_config(snap_ref, 'dm', dm)
Jun  8 08:10:41 xen5 SM: [8958]   File "/usr/lib/python2.7/site-packages/XenAPI.py", line 254, in __call__
Jun  8 08:10:41 xen5 SM: [8958]     return self.__send(self.__name, args)
Jun  8 08:10:41 xen5 SM: [8958]   File "/usr/lib/python2.7/site-packages/XenAPI.py", line 150, in xenapi_request
Jun  8 08:10:41 xen5 SM: [8958]     result = _parse_result(getattr(self, methodname)(*full_params))
Jun  8 08:10:41 xen5 SM: [8958]   File "/usr/lib/python2.7/site-packages/XenAPI.py", line 228, in _parse_result
Jun  8 08:10:41 xen5 SM: [8958]     raise Failure(result['ErrorDescription'])
Jun  8 08:10:41 xen5 SM: [8958]
Jun  8 08:10:41 xen5 SM: [8958] Raising exception [46, The VDI is not available [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']]]
Jun  8 08:10:41 xen5 SM: [8958] lock: released /var/lock/sm/c87fe31a-3a00-409b-af05-208c08904d8b/vdi
Jun  8 08:10:41 xen5 SM: [8958] ***** generic exception: vdi_activate: EXCEPTION <class 'SR.SROSError'>, The VDI is not available [opterr=['MAP_DUPLICATE_KEY', 'VDI', 'sm_config', 'OpaqueRef:82c42d87-4542-8169-20a4-0bdb52cd162b', 'dm']]
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/SRCommand.py", line 110, in run
Jun  8 08:10:41 xen5 SM: [8958]     return self._run_locked(sr)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/SRCommand.py", line 159, in _run_locked
Jun  8 08:10:41 xen5 SM: [8958]     rv = self._run(sr, target)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/SRCommand.py", line 264, in _run
Jun  8 08:10:41 xen5 SM: [8958]     writable, caching_params)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 1541, in activate
Jun  8 08:10:41 xen5 SM: [8958]     if self._activate_locked(sr_uuid, vdi_uuid, options):
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/blktap2.py", line 95, in wrapper
Jun  8 08:10:41 xen5 SM: [8958]     raise xs_errors.XenError(excType, opterr=msg)
Jun  8 08:10:41 xen5 SM: [8958]   File "/opt/xensource/sm/xs_errors.py", line 52, in __init__
Jun  8 08:10:41 xen5 SM: [8958]     raise SR.SROSError(errorcode, errormessage)
Jun  8 08:10:41 xen5 SM: [8958]
rposudnevskiy commented 7 years ago

Judging by the error RBDSR tries to add dm key to sm_config butsm_config` have such key already. So you used right approach. If you forget and reintroduce the vdi it should have clean metadata after that. Did the "forget/introduce" approach help you to resolve the problem?

nate-byrnes commented 7 years ago

That did it. (Forget, introduce that is).

It seems to me that whenever I force shutdown a guest, I need to perform this process of forget then introduce.... I'm still focused on some production repairs, so I haven't been able to thoroughly test this yet, so I thought I'd mention it in case it was something you were familiar with.

nate-byrnes commented 7 years ago

It seems pretty consistent that a force shutdown requires me to forget then introduce the VDI's of the machine shutdown. I think that the sm_config is not being re-set for whatever reason under force shutdown circumstances. That said, forget/introduce is now working, so, I'll close this issue.