Unable to move VDI from RBDSR to other SR

[nate-byrnes]

When I try to move an RBD-NBD VDI to another SR I am always getting:

Jan 29 11:05:02 xen3 SM: [29024] on-slave.is_open: {'vdiUuid': '939a5426-51aa-42f2-91bd-13ce4e1fa2e5', 'srRef': 'OpaqueRef:ddd35779-b212-ecf2-486d-83a0be957f9c'}
Jan 29 11:05:03 xen3 SM: [29024] Raising exception [48, Unknown repository type]
Jan 29 11:05:03 xen3 SM: [29024] ***** is_open: EXCEPTION <class 'SR.SROSError'>, Unknown repository type
Jan 29 11:05:03 xen3 SM: [29024]   File "/etc/xapi.d/plugins/on-slave", line 97, in is_open
Jan 29 11:05:03 xen3 SM: [29024]     return _is_open(session, args)
Jan 29 11:05:03 xen3 SM: [29024]   File "/etc/xapi.d/plugins/on-slave", line 86, in _is_open
Jan 29 11:05:03 xen3 SM: [29024]     driver = SR.driver(srType)
Jan 29 11:05:03 xen3 SM: [29024]   File "/opt/xensource/sm/SR.py", line 64, in driver
Jan 29 11:05:03 xen3 SM: [29024]     raise xs_errors.XenError('SRUnknownType')
Jan 29 11:05:03 xen3 SM: [29024]   File "/opt/xensource/sm/xs_errors.py", line 52, in __init__
Jan 29 11:05:03 xen3 SM: [29024]     raise SR.SROSError(errorcode, errormessage)
Jan 29 11:05:03 xen3 SM: [29024]

[nate-byrnes]

This is running the latest v1.0 downloaded as a zip as of today.

[nate-byrnes]

After looking further into this, I am curious if VDI's created with prior versions of the RBDSR are incompatible with the current version. I just tried using XenOrchestra to move the RBD VDI to another SR, and in going through the traceback I found that a field named "dev_instance" is expected in the sm-config field of the VDI. This field does not exist on my older (1yr or so I'd imagine) VDI's.

[nate-byrnes]

Presumably because I had a pretty old install of RBDSR, my metadata was somewhat incompatible with the more recent release. Through a bunch of troubleshooting it seems that there were some issues with my sm-config data. I have been seeing key-errors in my SMlog, and have tweaked the code in ceph_utils.py to try to deal with some of these. I also needed to copy RBDSR to RBDSR.py in /opt/xensource/sm/ as well, and import RBDSR in /etc/xapi.d/on-slave's is_open function. With these changes and the patch below, I was able to forget and reintroduce my VDI's to then be able to export them to XVA's and import them to my alternate SR. I had to go through the re-introducing step because many of my VDI's ended up with "paused: True" in the sm-config, which was preventing them from being operated upon. It seems that some of the earlier errors I was seeing, coupled with the SR config to update_on_scan, was causing many of my VDI's to get set to paused, and then pretty much locked out from any subsequent activities.

--- RBDSR-1.0/bins/cephutils.py 2018-01-19 19:42:22.000000000 -0500
*************** class VDI:
*** 478,488 ****
          else:
              dm = "none"

!         if sm_config.has_key("dev_instance"):
!             dev = sm_config["dev_instance"]
!         else:
!             util.SMlog("sm_config has no entry dev_instance in {}. Imputing it....")
!             dev="0"

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"
--- 478,484 ----
          else:
              dm = "none"

!         dev = sm_config["dev_instance"]

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"
*************** class VDI:
*** 501,508 ****
          self._call_plugin('_map',args)

      def __unmap_VHD(self, vdi_uuid):
-         vdi_name = "%s" % (vdi_uuid)
-         dev_name = "%s/%s" % (self.sr.SR_ROOT, vdi_name)
          _vdi_name = "%s%s" % (VDI_PREFIX, vdi_uuid)
          _dev_name = "%s/%s" % (self.sr.DEV_ROOT, _vdi_name)
          _dmdev_name = "%s%s" % (self.sr.DM_ROOT, _vdi_name)
--- 497,502 ----
*************** class VDI:
*** 523,535 ****
          util.SMlog("Calling cephutills.VDI._unmap_VHD: vdi_uuid=%s, dm=%s, sharable=%s" % (vdi_uuid, dm, sharable))

          args = {"mode":self.mode, "vdi_uuid":vdi_uuid,
!                 "_vdi_name":_vdi_name,  "_dev_name":_dev_name, "dev_name": dev_name,
                  "_dmdev_name":_dmdev_name, "_dm_name":_dm_name,
                  "CEPH_POOL_NAME":self.sr.CEPH_POOL_NAME,
                  "NBDS_MAX":str(NBDS_MAX),
                  "CEPH_USER":self.sr.CEPH_USER,"sharable":sharable,
                  "dm":dm}
!         self._call_plugin('unmap',args)

      def _map_VHD(self, vdi_uuid, size, dm):
          _vdi_name = "%s%s" % (VDI_PREFIX, vdi_uuid)
--- 517,529 ----
          util.SMlog("Calling cephutills.VDI._unmap_VHD: vdi_uuid=%s, dm=%s, sharable=%s" % (vdi_uuid, dm, sharable))

          args = {"mode":self.mode, "vdi_uuid":vdi_uuid,
!                 "_vdi_name":_vdi_name,  "_dev_name":_dev_name,
                  "_dmdev_name":_dmdev_name, "_dm_name":_dm_name,
                  "CEPH_POOL_NAME":self.sr.CEPH_POOL_NAME,
                  "NBDS_MAX":str(NBDS_MAX),
                  "CEPH_USER":self.sr.CEPH_USER,"sharable":sharable,
                  "dm":dm}
!         self._call_plugin('_unmap',args)

      def _map_VHD(self, vdi_uuid, size, dm):
          _vdi_name = "%s%s" % (VDI_PREFIX, vdi_uuid)
*************** class VDI:
*** 541,551 ****

          vdi_ref = self.session.xenapi.VDI.get_by_uuid(vdi_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(vdi_ref)
!         if sm_config.has_key("dev_instance"):
!             dev = sm_config["dev_instance"]
!         else:
!             util.SMlog("sm_config has no entry dev_instance in {}. Imputing it....")
!             dev="0"

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"
--- 535,541 ----

          vdi_ref = self.session.xenapi.VDI.get_by_uuid(vdi_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(vdi_ref)
!         dev = sm_config["dev_instance"]

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"
*************** class VDI:
*** 611,621 ****

          snap_ref = self.session.xenapi.VDI.get_by_uuid(snap_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(snap_ref)
!         if sm_config.has_key("dev_instance"):
!             dev = sm_config["dev_instance"]
!         else:
!             util.SMlog("sm_config has no entry dev_instance in {}. Imputing it....")
!             dev="0"

          if self.session.xenapi.VDI.get_sharable(snap_ref):
              sharable="true"
--- 601,607 ----

          snap_ref = self.session.xenapi.VDI.get_by_uuid(snap_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(snap_ref)
!         dev = sm_config["dev_instance"]

          if self.session.xenapi.VDI.get_sharable(snap_ref):
              sharable="true"
*************** class VDI:
*** 684,694 ****
          vdi_ref = self.session.xenapi.VDI.get_by_uuid(vdi_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(vdi_ref)
          dm="mirror"
!         if sm_config.has_key("dev_instance"):
!             dev = sm_config["dev_instance"]
!         else:
!             util.SMlog("sm_config has no entry dev_instance in {}. Imputing it....")
!             dev="0"

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"
--- 670,676 ----
          vdi_ref = self.session.xenapi.VDI.get_by_uuid(vdi_uuid)
          sm_config = self.session.xenapi.VDI.get_sm_config(vdi_ref)
          dm="mirror"
!         dev = sm_config["dev_instance"]

          if self.session.xenapi.VDI.get_sharable(vdi_ref):
              sharable="true"