Can't enable HA.

[mstarikov]

iSCSI SR uses dedicated class to enable HA(LVHDoISCSIVDI) and tests if iscsid daemon is running and initiator name is set with "localIQN" parameter.

This key is not applicable to RBD connection, since we only need rbd driver to be loaded into the kernel to be able to establish connection to CEPH cluster.

Following patch(no spaces in the patch preview, RBD-HA.patch.zip attached) would workaround the problem, but need to test if there are any other uses of localIQN in HA. RBD-HA.patch.zip

--- /opt/xensource/sm/LVHDoISCSISR.py 2016-05-09 14:48:38.000000000 +1000 +++ ./LVHDoISCSISR.py 2016-05-09 14:48:19.000000000 +1000 @@ -552,7 +552,10 @@ if not lvutil._checkLV(self.path): raise xs_errors.XenError('VDIUnavailable') dict = {} - self.sr.dconf['localIQN'] = self.sr.iscsi.localIQN + if self.sr.dconf['port'] in '6789': + self.sr.dconf['localIQN'] = 'rbd-client' + else: + self.sr.dconf['localIQN'] = self.sr.iscsi.localIQN self.sr.dconf['multipathing'] = self.sr.mpath self.sr.dconf['multipathhandle'] = self.sr.mpathhandle dict['device_config'] = self.sr.dconf

[tomzpl]

Hi, I patched Xenserver 7 with above patch, but I cannot add SR to the repository: [root@XEN-SRV-01 rbdsr-master]# xe sr-create type=lvmoiscsi name-label=CEPH_RBD_CLI shared=true device-config:target=XEN-SRV-00 device-config:port=6789 device-config:targetIQN=rbd device-config:SCSIid=rbd_CEPH-SR device-config:chapuser=ceph device-config:chappassword=password There was an SR backend failure. status: non-zero exit stdout: stderr: File "/opt/xensource/sm/LVMoISCSISR", line 586 self.sr.dconf['localIQN'] = 'rbd-client' ^ IndentationError: expected an indented block

I'm successful added the SR before using GUI or CLI without patch, but HA was not working:

Enabling HA on pool 'XEN-SRV-00' This operation cannot be performed because VDI OpaqueRef:d2fe32ac-4de6-0cc7-3773-d7c32cace11f could not be properly attached to the VM. XEN-SRV-00 Aug 6, 2016 1:51 PM Dismiss

Thanks for the driver!

[mstarikov]

Hi Tom,

Did you patch it with the file inside of the RBD-HA.patch.zip or with copy-paste from the comment?

Like I've said, the comment doesn't have proper indent(the github formatting thing) so it will throw an error if you've used comment version.

You can reverse the patch and then apply one from the zip or you can open LVMoISCSISR with vim or nano and insert 8 spaces before statement on line 586 and probably 587,8,9.

If you having problems with that, email me the LVMoISCSISR and I'll have a look

[tomzpl]

Sorry being so late but everything is working fine, I had to apply the patch which added: self.localIQN = 'rados_sr' to RBDSR.py.

With this patch only I could move VMs and recover them after node failure.

I have put steps on my blog: http://tomz.pl/virtualization/86-xenserver/104-xenserver-7-ha-cluster-with-ceph

how to create the enviorment,

Thank you for the great work and providing us awesome driver!

Regards,

[mstarikov]

Hi Tom,

That's great news. I just have few suggestions about your instructions if I may:

1. It's not a good idea to run HA in 2 host pool. Like MONs in CEPH, HA in XS relies on the comparison of partition size to determine which host to fence. If partition is split 50/50 there is no reliable way for HA to determine which host has failed. See here for the details: http://support.citrix.com/article/CTX129721
2. If you are installing CEPH components in Dom0, I don't think you need RBD plugin. Perhaps you could use systemd start up script to attach rbd object on boot and create local LVM SR on top... not really sure if that will work, but main point of this SR plugin in my mind was to avoid changes of Dom0 and maintain supportability/upgradability if necessary. Based on that, in the article here, I was deploying guest that would run as CEPH nodes and plug it back to the Dom0 on boot: XENSERVER + DOCKER + CEPH/RBDSR FOR SHARED LOCAL STORAGE

Other than that, excellent stuff. I appreciate your interest in the plugin! :)

One of those days I will merge the localIQN changes into the RBDSR.py. I just need to test and recreate patches, because that one line would shift everything around.